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ABSTRACT 


This  thesis  investigates  the  relationship  between 
observable  characteristics  and  performance  during  the  primary 
phase  of  flight  training.  The  data  for  this  study  consists  of 
272  observations  from  Naval  Academy  graduates  in  the  classes 
of  1995  and  1996.  Analysis  of  the  variables  was  conducted 
using  the  Heckman  two-stage  regression  technique  to  correct 
for  possible  selectivity  bias.  In  this  technique  the  first- 
stage  probit  model,  which  predicts  the  likelihood  of  primary 
phase  completion,  is  used  to  generate  a  correction  factor  for 
possible  selectivity  bias.  The  correction  factor  is  then  used 
in  the  second-stage  adjusted  least-squares  regression  model. 
The  conclusions  from  this  study  are:  The  biographical 
inventory  from  the  Aviation  Selection  Test  Battery (ASTB)  is  a 
valid  predictor  of  primary  phase  completion.  The  Pilot  Flight 
Aptitude  Rating  (PFAR)  from  the  ASTB,  academic 
achievement  (AQPR)  at  the  Naval  Academy,  and  previous  flight 
experience  are  valid  predictors  of  flight  training 
performance.  Additionally,  it  appears  that  sample  selection 
bias  does  not  seem  to  be  a  problem  in  this  analysis. 
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I .  INTRODUCTION 


The  selection  of  naval  aviators  is  serious  business. 

The  training  program  is  a  long,  arduous,  and  expensive 
process,  often  requiring  in  excess  of  two  years  to  complete. 
The  inherent  difficulty  of  the  program  is  necessitated  by 
the  demands  that  will  eventually  be  placed  upon  these 
individuals.  Navy  and  Marine  Corps  pilots  are  required  to 
operate  under  conditions  that  are  largely  unique  to  the  sea¬ 
going  services  such  as  shipboard  operations  where  the 
nearest  alternate  landing  sight  could  be  several  thousand 
miles  away.  Despite  these  demands,  naval  aviators  continue 
to  operate  around  the  world  conducting  day-and-night  flight 
operations  in  conditions  that  would  cause  most  prudent 
civilian  aviators  to  remain  safely  on  the  ground. 

The  sudden  collapse  of  the  Soviet  Union  in  the  early 
1990 's  brought  to  an  end  the  high  levels  of  military 
spending  that  had  been  sustained  during  the  Cold  War  era. 
Lacking  a  specific  foe  to  concentrate  on,  the  perceived 
threat  from  external  forces  diminished  and  the  American 
public  called  for  a  "peace  dividend."  In  response  to  public 
opinion.  Congress  continues  to  support  reduced  levels  of 
military  spending.  The  military  budget  for  1997  was  set  at 
around  265  billion  dollars,  and  as  we  approach  the  turn  of 
the  century,  will  remain  at  this  level,  with  only  marginal 
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increases  and  zero  real  growth  in  spending  for  the 
foreseeable  future.  Reduced  budgets  have  not  necessarily 
resulted  in  reduced  commitments.  Military  leaders  are  still 
required  to  plan,  equip,  and  train  forces  to  fight  two 
simultaneous  Major  Theater  wars.  The  implications  for 
military  leaders,  given  these  budgetary  constraints,  are 
that  they  need  to  ensure  maximum  return  on  every  dollar 
spent . 

A.  INVESTMENTS  IN  CAPITAL 

Navy  leaders  must  seek  to  optimize  investments  in  both 
physical  and  personnel  capital  in  order  to  maximize  military 
effectiveness.  Investments  in  military  physical  capital 
include  ships,  aircraft,  and  bases.  Effectiveness  can  be 
maximized  by  examining  which  is  the  most  appropriate  weapons 
system,  which  system  has  the  lowest  initial  acquisition 
cost,  and  which  system  has  the  lowest  life  cycle  operating 
and  support  cost.  Investments  in  military  physical  capital 
are  highly  political  and  are  often  driven  by  forces  beyond 
the  control  of  Navy  leaders.  Investments  in  military 
personnel  capital  take  the  form  of  educating  or  training 
personnel  to  enhance  performance.  Examples  of  investments 
in  human  capital  include  post-graduate  education  and  Navy 
training  schools  such  as  nuclear  power  school  and  flight 
training. 
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Flight  training  is  one  of  the  most  expensive  training 
opportunities  provided  by  the  Navy.  The  training  costs  per 
individual  range  from  $350,000  for  a  helicopter  pilot  to 
$700,000  for  a  jet  pilot  (Bowman,  1995).  Bowman  points  out 
that  these  estimates  do  not  take  into  consideration  the 
costs  of  the  military  flight  instructors,  so  the  actual 
costs  are  much  higher.  An  additional  training  cost 
associated  with  these  individuals  is  the  cost  to  make 
selected  pilots  proficient  in  a  particular  warfare 
specialty.  This  additional  training  requires  a  substantial 
investment  of  both  resources  and  time. 

The  process  of  pilot  selection  can  help  maximize  the 
Navy' s  investment  in  personnel  capital  by  selecting  those 
individuals  with  the  highest  probability  of  completing 
flight  training,  thereby  reducing  the  high  costs  associated 
with  attrition  from  the  flight  program.  The  problem  of 
attrition  from  flight  training  is  neither  a  new  phenomenon, 
nor  is  it  unique  to  the  Navy.  It  exists  for  every  military 
organization  that  conducts  flight  training  and  is  a  major 
concern  because  the  attrition  cost  per  individual  increases 
as  the  time  spent  in  the  program  increases.  The  cost  has 
recently  been  estimated  at  $18,000  for  an  individual  who 
attrites  from  Aviation  Pre-flight  Indoctrination  (API), 
$50,000  while  in  Primary,  $200,000  for  Advanced  Helicopter, 
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and  up  to  $500,000  for  individuals  in  Advanced  Jet 
training (Blower,  1997).  Attrition  has  the  additional  side 
effect  of  hindering  the  Training  Command's  ability  to  meet 
the  manning  requirements  of  fleet  squadrons.  If  attrition 
is  higher  than  expected,  a  serious  shortfall  in  the 
operational  manning  level  of  the  fleet  squadrons  could 
develop,  resulting  in  reduced  effectiveness  and  a  weakening 
of  national  defense. 

In  addition  to  the  direct  cost  of  attrition,  the 
opportunity  cost  of  training  opportunities  lost  to  those  who 
attrite  must  be  accounted  for  when  assessing  the  total  cost 
involved  in  a  flight  failure.  Consider  the  following 
hypothetical  example.  A  Midshipman  selected  for  aviation 
training  graduates  from  the  Naval  Academy  and  is  given  a  six 
month  temporary  assignment  while  waiting  to  report  to 
Pensacola  to  begin  flight  training.  The  individual  reports 
to  Pensacola,  completes  Aviation  Indoctrination  and  begins 
flight  training. 

Unfortunately,  this  student  attrites  in  the  14’^*'  week 
of  primary  training.  This  individual  is  redesignated  and 
sent  to  another  community (e . g. ,  Surface  Line  or  Supply 
Corps)  to  begin  training  anew.  If  this  flight  failure  had 
been  predicted,  the  student  in  question  could  have  been  sent 
directly  to  a  different  community  for  training  and  would 
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have  reported  to  the  fleet  to  relieve  another  officer  almost 
a  year  earlier. 

As  can  be  seen  by  looking  at  the  bigger  picture,  the 
costs  associated  with  a  flight  failure  are  actually  much 
higher  than  simply  the  resources  expended  on  a  particular 
individual  while  in  the  flight  program.  The  research 
question  to  be  addressed  in  this  thesis  is:  Are  there 
identifiable  and  significant  characteristics  that  can  be 
used  to  improve  the  pilot  selection  process  in  use  at  the 
Naval  Academy? 

B .  SCOPE 

The  scope  of  this  research  project  is  limited  to  Naval 
Academy  graduates  from  the  classes  of  1995  and  1996.  To 
date,  1995  and  1996  represent  the  only  classes  that  have 
taken  the  latest  version  of  the  Aviation  Selection  Test 
Battery (ASTB)  for  which  flight  training  performance  data  are 
available  and  have  been  subject  to  the  new  service 
assignment  policy.  Graduates  from  these  classes  who  were 
selected  for  student  naval  flight  officer  training  were  not 
included  due  to  differences  in  the  training  curriculum 
following  Aviation  Indoctrination.  There  was  no  attempt  to 
include  peer  student  pilots  from  other  accession  sources, 
who  had  also  taken  the  latest  version  of  the  ASTB,  due  to 
the  extreme  difficulty  in  collecting  relevant  data. 
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Data  for  this  thesis  were  compiled  from  a  variety  of 
sources  including  the  Naval  Operational  Medicine 
Institute (NOMI) ,  Pensacola  Florida,  Naval  Personnel  Records 
Data  Center,  San  Diego,  and  Institutional  Research,  United 
States  Naval  Academy.  Data  sets  were  combined  using  Excel 
spreadsheets  and  all  statistical  analysis  was  conducted 
using  SPSS  7.5. 

C .  HYPOTHESES 

This  research  paper  will  test  the  following  hypotheses: 

•  Naval  Academy  graduates  who  score  higher  on  the 
biographical  inventory  are  more  likely  to  complete 
primary  flight  training  than  those  with  lower 
scores . 

•  Naval  Academy  graduates  who  score  higher  on  the 
pilot  flight  aptitude  rating  achieve  higher  flight 
grades  that  those  with  lower  scores. 

•  Naval  Academy  graduates  with  higher  academic 
quality  point  ratings  achieve  higher  flight  grades 
than  those  with  lower  ratings. 

•  Naval  Academy  graduates  with  higher  military 
quality  point  ratings  achieve  higher  flight  grades 
than  those  with  lower  ratings. 

•  Naval  Academy  graduates  with  technical  degrees 
achieve  higher  scores  that  graduates  with  non¬ 
technical  degrees. 

•  Male  graduates  achieve  higher  flight  grades  than 
female  graduates. 

•  Ethnic  majority  graduates  achieve  higher  flight 
grades  than  minority  graduates . 

•  Naval  Academy  graduates  with  previous  flight 
experience  achieve  higher  flight  grades  than 
graduates  with  no  previous  flight  experience. 


6 


D .  ORGANIZATION 


Following  the  introduction.  Chapter  II  will  look  at 
various  pilot  selection  methods  used  by  the  Navy,  Air  Force 
and  a  selection  of  other  organizations  as  a  means  of 
acguainting  the  reader  with  the  numerous  and  varied  methods 
utilized  in  selecting  pilot  candidates.  Chapter  III  will 
present  a  history  of  pilot  selection.  Chapter  IV  will 
discuss  the  methodology  and  models  to  be  used  in  analyzing 
the  data.  Chapter  V  will  present  and  interpret  the  results 
of  the  models.  Finally,  Chapter  VI  will  summarize  the 
conclusions  of  this  study  and  offer  recommendations  for 
future  research. 
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II.  PILOT  SELECTION  METHODS 


A.  NAVY 

The  Navy  has  several  sources  from  which  to  draw 
potential  student  naval  aviators;  the  primary  three  are 
Officer  Candidate  School (OCS) ,  Reserve  Officer  Training 
Corps (ROTC),  and  the  Naval  Academy (USNA) .  Regardless  of 
accession  source  all  student  naval  aviators  must  meet 
certain  basic  requirements  which  include: 

•  Hold  an  accredited  bachelor's  degree. 

•  Be  at  least  19  but  less  than  27  years  of  age  at 
the  time  of  commissioning. 

•  Be  physically  qualified  and  aeronautically 
adaptable  according  to  Navy  standards,  with  20/20 
uncorrected  vision  and  normal  color  and  depth 
perception. 

•  Meet  anthropometric  standards  which  determine  if 
the  aviator  can  safely  fit  in  an  aircraft  cockpit. 

•  Achieve  a  qualifying  score  on  the  Aviation 
Selection  Test  Battery (ASTB) . 

1.  Aviation  Selection  Test  Battery 

The  1992  Navy  and  Marine  Corps  Aviation  Selection  Test 
Battery (ASTB)  is  the  most  recent  revision  in  a  series  of 
cognitive  pencil-and-paper  tests  that  have  been  used  by  the 
naval  services  as  a  selection  measure  for  student  naval 
aviators.  This  test  battery  replaced  the  Academic 


9 


Qualifying  Test/Flight  Aptitude  Rating (AQT/FAR)  that  was 
first  instituted  in  1942  with  revisions  in  1953  and  1971. 
Reasons  for  the  latest  revision  include:  changes  in 
demographics  due  to  the  all-volunteer- force,  changes  in 
naval  aviation  training,  decrease  in  predictive  validity  of 
the  previous  test,  possible  compromise  of  the  test  since  its 
last  revision,  and  changes  in  federal  guidelines  regarding 
employee  selection  procedures. 

Developed  by  Educational  Testing  Services  of  Princeton, 
New  Jersey,  in  conjunction  with  the  Naval  Aerospace  and 
Operational  Medical  Institute,  the  ASTB  is  composed  of  six 
subtests:  Math-Verbal  Test(MVT),  Mechanical  comprehension 
Test(MCT),  Spatial  Apperception  Test (SAT),  Aviation  and 
Nautical  Test (AN) ,  Biographical  Inventory (BI) ,  and  Aviation 
Interest  (AI ) .  Weighted  combinations  of  the  various  subtests 
result  in  five  stanine  scores (a  standard  score  on  a  nine 
point  scale) ,  that  are  used  to  predict  attrition,  academic 
performance,  and  basic  flight  performance.  These  scores 
are:  Academic  Qualifications  Rating (AQR) ,  Pilot  Flight 
Aptitude  Rating (PFAR) ,  Flight  Officer  Aptitude 
Rating (FOFAR) ,  Pilot  Biographical  Inventory  (PBI),  and 
Flight  Officer  Biographical  Inventory (FOBI) . 

The  ASTB  differs  from  previous  tests  in  that  only  the 
PBI  and  FOBI  are  intended  to  predict  attrition.  The  AQR 
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predicts  academic  performance,  and  the  PFAR  and  FOFAR 
predict  basic  flight  performance (Frank  &  Baisden,  1993) . 

Navy  pilot  applicants  must  obtain  qualifying  scores  of  3/4/4 
on  the  AQR,  PFAR,  and  BI.  Marine  pilots  must  achieve 
scores  of  4/6/4.  Once  these  basic  requirements  have  been 
met,  each  accession  source  uses  a  unique  process  to  fill  its 
allotment  of  primary  flight  training  billets. 

2 .  Naval  Academy 

For  over  150  years  the  Naval  Academy  has  been  a  primary 
source  for  career  officers.  The  percentage  of  naval 
officers  who  are  academy  graduates  has  fallen  dramatically 
since  WWII  and  the  inception  of  the  ROTC  and  OCS  programs, 
but  the  academy  remains  the  largest  single  commissioning 
source  accounting  for  30%  of  the  pilot  community (Bowman, 

1995) .  The  historical  overall  attrition  rate  from  Navy 
flight  training  has  been  near  30%  but  Naval  Academy 
graduates  have  faired  slightly  better  than  student  naval 
aviators  from  the  other  two  accession  sources (North  & 
Griffin,  1977) .  This  has  been  attributed  to  the  fact  that 
the  selective  nature  of  the  admissions  process  combined  with 
a  rigorous  four  year  course  of  study  acts  as  a  effective 
filter  to  screen  for  adaptability  to  a  military  environment 
(Griffin  &  Mosko,  1977) . 
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Admission  to  the  Naval  Academy  is  a  long  and  selective 
process  usually  involving  a  congressional  nomination  and 
what  is  known  as  the  whole-person  philosophy  which  generates 
a  candidate  multiple  based  on  academic,  athletic,  and  civic 
accomplishments.  While  at  the  Naval  Academy,  Midshipmen 
undergo  a  rigorous  four  year  academic  and  military 
curriculum.  The  culmination  of  these  four  years  is  service 
assignment.  Prior  to  1995  Midshipmen  selected  their  desired 
community  based  solely  on  Order  of  Merit  and  physical 
qualification.  The  Order  of  Merit  was  determined  by 
academic,  military  performance,  and  conduct  grades,  with 
academic  grades  the  primary  determinant. 

The  new  selection  process  involves  an  interview  phase, 
community  screening  phase,  preference  designation  phase,  and 
an  assignment  phase  that  is  designed  to  make  the  process 
even  more  selective.  During  the  interview  phase  each 
graduating  Midshipman  is  interviewed  by  a  team  consisting  of 
two  or  three  commissioned  officers  from  the  different 
service  communities.  The  interview  is  then  scored  on  a 
scale  of  zero  to  ten,  with  zero  being  unsatisfactory  and  ten 
representing  excellent,  in  five  areas  including  appearance 
and  poise,  oral  communication,  leadership  potential,  service 
community  motivation,  and  service  community  understanding. 

The  community  screening  phase  is  designed  to  assess 
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medical  qualification  and  community  specific  academic  and 
physical  requirements.  For  potential  student  naval  aviators 
this  requirement  includes  the  minimum  score  on  the  Aviation 
Selection  Test  Battery (ASTB)  described  earlier.  Preference 
designation  phase  is  the  process  during  which  each  First- 
class  Midshipman  will  officially  indicate  their  service  and 
community  preferences  from  among  those  which  they  have  been 
found  qualified.  During  the  final  assignment  phase,  service 
assignment  boards  from  each  community  will  select  the  best 

qualified  Midshipmen  from  among  those  applying. 

B.  AIR  FORCE 

The  Air  Force  operates  a  pilot  training  program  similar 
to  that  of  the  Navy.  They  also  rely  on  three  primary 
accession  sources,  the  Air  Force  Academy,  AFROTC,  and 
Officer  Training  School,  to  provide  student  pilots  for 
Specialized  Undergraduate  Pilot  Training (SUPT) .  The 
requirements  for  SUPT  are  that  the  candidate  be  less  than  27 
and  'h.  years  old,  have  a  bachelor's  degree,  pass  a  physical 
examination,  obtain  satisfactory  scores  on  the  Air  Force 
Officer  Qualifying  Test(AFOQT)  and  Basic  Attributes 
Test (BAT),  and  successfully  complete  a  flight  screening 
program. 
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1 .  Air  Force  Officer  Qualifying  Test 

The  Air  Force  Officer  Qualifying  Test(AFOQT)  is  a 
paper-and-pencil  multiple  aptitude  battery  similar  to  the 
Navy  ASTB.  This  test  is  used  for  officer  commissioning  and 
aircrew  training  selection.  The  test  has  been  in  use  since 
1957,  with  new  forms  being  developed  about  every  seven 
years.  The  current  version  of  the  AFOQT  is  Form  0. 

The  current  AFOQT  battery  has  five  composite  scores: 
Verbal,  Quantitative,  Academic  Aptitude,  Pilot,  and 
Navigator-Technical.  Its  overall  purpose  is  to  test  general 
cognitive  ability, (g) ,  and  five  lower  order  functions: 
verbal,  math,  spatial,  interest/aptitude,  and  perceptual 
speed.  The  16  tests  included  in  battery  are:  Verbal 
Analogies (VA) ,  Arithmetic  Reasoning (AR) ,  Reading 
Comprehension (RC) ,  Data  Interpretation (DI) ,  Word 
Knowledge (WK) ,  Math  Knowledge (MK) ,  Mechanical 
Comprehension (MC) ,  Electrical  Maze (EM) ,  Scale  Reading (SR), 
Instrument  Comprehension (IC)  ,  Block  Counting (BC) ,  Table 
Reading (TR),  Aviation  Information (AI) ,  Rotated  Blocks (RB) , 
General  Science(GS),  and  Hidden  Figures (HF) (Carretta,  1997). 

The  Basic  Attributes  Test  (BAT)  is  a  computer-based  test 
battery  used  to  augment  the  results  of  the  AFOQT  for  pilot 
selection.  It  consists  of  five  tests  that  measure 
psychomotor  coordination,  short-term  memory,  and  attitudes 
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towards  risk  taking.  The  BAT  tests  are  the  Two-hand 
Coordination,  Complex  Coordination,  Item  Recognition,  Time 
Sharing,  and  Activities  Interest  Inventory. 

2.  Air  Force  Academy 

The  Air  Force  Academy  located  in  Colorado  Springs, 
Colorado,  is  in  many  ways  comparable  to  the  Naval  Academy. 
It  is  a  four  year  institution  that  provides  students  with 
total  immersion  in  a  military  culture  and  has  a  similarly 
competitive  admissions  process  that  normally  requires  a 
congressional  nomination.  Approximately  65  percent  of  each 
Air  Force  Academy  graduating  class  will  enter  Specialized 
Undergraduate  Pilot  Training (SUPT) .  This  represents 
approximately  25  percent  of  all  pilot  candidates  who  enter 
SUPT  each  year.  During  the  course  of  instruction  at  the 
Academy,  cadets  participate  in  the  Pilot  Indoctrination 
Program (PIP) . 

The  stated  purpose  of  this  program  is  "to  identify 
students  who  possess  the  potential  to  complete  SUPT  and 
motivate  qualified  Academy  graduates  toward  a  rated  career 
in  the  Air  Force."  This  program  consists  of  seven  hours  of 
airmanship  academics  and  20  hours  of  light  single  engine 
training.  PIP  also  has  an  optional  phase  which  consists  of 
4.5  hours  of  incentive  sorties.  Upon  graduation  each 
candidate  who  has  successfully  completed  PIP,  is  medically 
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qualified,  receives  a  positive  recommendation  from  a  flying 
supervisor,  and  has  a  desire  to  attend  pilot  training  will 
receive  a  SUPT  assignment.  Air  Force  Academy  graduates  are 
exempted  from  taking  the  Air  Force  Officer  Qualifying 
Test (AFOQT) . 

3.  Air  Force  Reserve  Officer  Training  Corps 

Air  Force  Reserve  Officer  Training  Corps (AFROTC) 
provides  the  largest  number  of  aviation  candidates, 
accounting  for  over  one- third  of  SUPT  entrants.  ROTC 
candidates  come  from  over  150  colleges  and  universities  from 
across  the  United  States.  In  order  to  compete  for  an  ROTC 
SUPT  assignment,  candidates  must  pass  a  medical  evaluation 
and  achieve  a  minimum  score  on  the  AFOQT.  The  minimiam 
percentile  score  on  the  pilot  composite  is  a  25  and  10  on 
the  navigator-technical  composite.  In  addition,  the 
combined  score  must  total  at  least  50  (Lynch,  1991) .  Other 
factors  taken  into  consideration  are  grade  point  average. 
Scholastic  Aptitude  Test  scores,  Basic  Attributes  Test (BAT) 
score,  and  unit  commander  ratings.  Selection  for  ROTC  SUPT 
quotas  is  determined  by  a  central  selection  board  that 
convenes  semi-annually  at  Maxwell  AFB,  Alabama. 

Candidates  approved  by  the  selection  board  who  do  not 
already  possess  a  private  pilots  license  are  required  to 
complete  Light  Aircraft  Training (LATR)  between  their  junior 
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and  senior  year  at  the  USAF  Officer  Training  School  flight 
screening  facility  at  Hondo  AFB,  Texas.  LA.TR  includes  13 
hours  of  academic  instruction  and  14  hours  of  light  aircraft 
training.  Successful  completion  of  the  light  aircraft 
training  marks  the  end  of  the  flight  screening  phase  and 
candidates  enter  SUPT  following  graduation  and 
commissioning. 

4  .  Officer  Training  School 

Air  Force  Officer  Training  School (OTS)  is  a  120  day 
course  conducted  at  Lackland  AFB,  Texas,  that  provides  both 
officer  training  and  flight  screening.  OTS  production  rates 
are  more  flexible  than  the  Air  Force  Academy  or  the  ROTO 
program  and  are  therefore  used  as  a  buffer  to  accommodate 
changing  SUPT  production  rates.  Acceptance  as  a  cadet 
requires  a  college  degree,  passing  a  medical  examination, 
and  obtaining  a  qualifying  scores  on  the  AFOQT  and  BAT. 
Officer  Training  School  quotas  are  filled  by  the  OTS 
selection  board  that  is  convened  "as  required,"  currently 
six  times  per  year,  at  Randolph  AFB,  Texas. 

This  board  applies  what  is  known  as  the  Pilot  Candidate 
Selection  Method (PCSM)  to  rate  each  applicant.  The  PCSM 
uses  the  results  of  the  AFOQT,  BAT,  college  GPA,  and 
previous  flight  hours  to  generate  a  whole  person  score  for 
each  applicant.  The  highest  scoring  applicants,  up  to  the 
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quotas  available,  will  be  accepted  for  OTS.  Those  not 
selected  will  return  to  the  pool  to  await  the  next  selection 
board. 

OTS  candidates  who  do  not  already  possess  a  private 
pilot  license  are  sent  to  Hondo  AFB,  Texas  to  complete  the 
OTS  Flight  Screening  Program (FSP) .  Training  is  16  days  in 
length  and  includes  13  hours  of  academic  training  and  14 
hours  of  light  aircraft  training.  The  culmination  of  the 
course  is  a  solo  flight  and  a  final  evaluation  flight. 
Candidates  are  scored  as  either  satisfactory  or 
unsatisfactory.  Those  candidates  receiving  a  satisfactory 
score  begin  OTS  and  upon  completion  will  attend  SUPT. 

As  can  be  seen  from  the  descriptions  of  Navy  and  Air 
Force  selection  methods,  they  rely  heavily  on  the  use  of 
paper-and-pencil  general  cognitive  measures  to  select  pilot 
candidates.  The  major  difference  between  the  services  is 
that  the  Air  Force  places  an  additional  emphasis  on  light 
plane  screening  while  the  Navy  does  not.  In  the  remaining 
sections  of  this  chapter,  I  will  briefly  look  at  other  types 
of  selection  methods  in  use. 

C.  CANADIAN  AIR  FORCE 

The  pilot  selection  process  by  which  the  Canadian  Armed 
Forces  assesses  pilot  applicants  is  known  as  the  Canadian 
Automated  Pilot  Selection  System (CAPSS) .  CAPSS  is  a  stand- 
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alone  selection  devise  that  provides  a  measure  of  complex 
cognitive  abilities  and  psychomotor  coordination.  The 
underlying  constructs  that  CAPSS  is  designed  to  measure  are; 
psychomotor  coordination,  learning  rate,  multi-task 
integration,  and  performance  under-overload (Adams -Roy, 

1996)  . 

CAPSS  uses  flight  simulator  technology  to  collect  data 
on  a  sample  of  flight  skills  required  to  pilot  a  light 
single  engine  aircraft.  The  syllabus  consists  of  five  one- 
hour  sessions  where  each  segment  is  an  extension  of,  and 
uses  skills  learned  in,  the  previous  session.  At  the  end  of 
each  segment  a  score,  ranging  from  0  to  1  is  generated. 

This  score  represents  the  probability  of  a  candidate's 
successful  completion  of  flight  training.  The  best 
correlation  with  success  in  flight  training  has  come  from 
the  score  after  the  end  of  the  fourth  segment,  and  it  is 
this  score  which  is  used  to  select  pilot  candidates. 

D .  LUFTWAFFE 

The  German  Air  Force,  or  Luftwaffe,  uses  a  multi-stage 
psychological  selection  process  to  select  from  potential 
aviation  candidates.  This  process,  administered  by  Division 
IV  of  the  Luftwaffe  Institute  of  Aerospace  Medicine (FMI) , 
can  last  from  between  one  and  two  years .  The  division 
consists  of  four  branches:  Selection,  Screening, 


19 


Diagnostics,  and  Experimental  Psychology.  The  psychological 
selection  process  is  organized  into  three  sequential  steps: 
Pre-selection,  Main  Selection,  and  Flight  Screening.  Every 
applicant  must  successfully  complete  all  steps  to  be 
admitted  into  pilot  training (Gnan,  Flyer,  and  King,  1995). 

During  the  Pre-Selection  stage  any  male  who  wishes  to 
become  a  Lufftwaffe  pilot  may  apply.  The  average  age  of 
applicants  to  the  aviation  service  is  about  19.  Applicants 
will  first  take  the  Officer  Qualification  Test  which 
includes  objective  testing,  interviews  and  a  physical 
stamina  test.  Those  who  pass  the  test  will  be  sorted  into 
three  groups  through  the  use  of  an  intelligence  test  and  a 
concentration  test.  Applicants  from  the  top  two  groups  are 
accepted  for  the  next  selection  step.  The  lowest  performing 
group  may  be  accepted  in  cases  where  there  are  an 
insufficient  number  of  applicants  in  the  higher  performing 
groups . 

The  Main  Selection  stage  is  a  one-day  examination 
conducted  by  the  Selection  and  Diagnostic  Branches.  This 
examination  consists  of  three  computer-based  psychomotor 
test,  a  biographical  inventory,  and  a  diagnostic  interview. 
The  psychomotor  tests  measure  coordination,  selective 
attention,  reaction  time  and  accuracy,  multiple  task 
performance,  stress  tolerance,  and  decisiveness.  The 
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diagnostic  interview  focuses  on  stress  reaction  during  the 
tests,  coping  strategies,  achievement,  and  flying 
motivation.  When  the  tests  are  complete  the  results  are 
reviewed  by  a  board  of  psychologists  to  determine  if  the 
applicant  will  be  accepted  to  the  next  selection  step.  Once 
approved  by  the  selection  board,  candidates  are  given  a 
flight  physical  medical  evaluation.  During  the  Main 
Selection  phase  almost  30%  of  the  applicants  are  rejected. 
Having  passed  Pre-Selection,  Main  Selection,  and  physical 
screening,  applicants  will  be  sent  to  Primary  Military 
Training  and  Officer's  Training  School  before  beginning  the 
final  phase  of  Flight  Screening. 

Flight  Screening  begins  on  the  FPS-80  simulator  devise. 
Applicants  will  fly  several  missions  while  their  performance 
and  behavior  is  evaluated.  This  is  followed  by  70  hours  of 
academic  training  covering  navigation,  meteorology, 
aerodynamics,  and  airmanship.  The  final  step  in  the 
screening  process  is  18  hours  of  flight  time  in  a  light 
single  engine  aircraft,  culminating  in  a  solo  flight. 

Having  successfully  completed  all  stages  of  the  selection 
process,  candidates  will  be  sent  to  either  EURO-NATO  Joint 
Jet-Pilot  Training  at  Sheppard  AFB,  Texas,  or  Weapons 
Systems  Officer  training  at  Randolph  AFB,  Texas,  depending 
on  the  amount  of  points  accumulated  during  Flight  Screening. 
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E. 


LUFTHANSA 


The  Department  of  Aviation  and  Space  Psychology  of  the 
German  Aerospace  Research  Establishment (DLR)  selects  pilot 
applicants  for  ab-inito,  no  previous  flying  experience 
required,  training  for  Lufthansa  Airlines.  Pilot  applicants 
must  have  completed  the  German  university  entrance 
level (Abitur) ,  and  not  be  more  than  27  years  of  age.  The 
selection  process  is  completed  in  two  stages  that  evaluate 
flying  and  managerial  skills. 

The  first  stage  lasts  two  days  and  applicants  are 
required  to  perform  group  administered  performance  and 
personality  tests  designed  to  measure:  technical  knowledge 
and  comprehension,  mathematics,  concentration,  speed 
perception,  spatial  orientation,  and  temperament 
characteristics.  Personality  assessment  is  done  through  the 
Temperament  Structure  Scales (TSS)  designed  by  the  DLR 
specifically  for  pilot  selection.  The  TSS  is  a 
multidimensional  personality  questionnaire  that  reports  10 
personality  dimensions  measured  by  stanine  scales:  work- 
related  traits (motivation,  rigidity,  mobility,  and 
vitality),  social-behavior  traits (extroversion,  dominance, 
and  aggressiveness),  and  stress  resistance  factors (emotional 
stability,  spoiltness,  and  empathy) (Gnan,  et  al.,  1995). 

This  phase  of  screening  usually  reduces  the  number  of 
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applicants  by  70-75%. 

The  second  phase  lasts  three  days  and  consists  of 
psychomotor  coordination  and  multiple  task  capacity  tests, 
an  interview  in  front  of  a  selection  board,  and  a  medical 
examination.  Applicants  who  pass  the  second  phase,  usually 
about  10%  of  the  original  group,  are  sent  to  two  years  of 
pilot  training  in  either  Bremen,  Germany  or  Phoenix, 

Arizona.  Upon  completion  of  training  the  student  holds  all 
relevant  airline  licenses  including  the  Airline  Transport 
Pilot  License (ATP) .  After  a  five  month  transition  period 
that  includes  a  type  rating  and  a  line  check,  the  new  pilot 
is  certified  to  fly  as  first  officer  in  Lufthansa's  entry 
fleet  of  Boeing  737s. 

F.  AMERICAN  CIVILIAN  CARRIERS 

Lufthansa  differs  from  most  large  American  air  carriers 
in  that  most  of  their  pilots  have  no  previous  flight 
experience  and  are  trained  in-house.  For  decades  American 
air  carriers  have  relied  on  hiring  surplus  military  pilots 
with  significant  experience  and  training  in  either  the  Navy 
or  Air  Force.  The  luxury  of  being  able  to  fill  their  ranks 
with  military  trained  pilots  is  rapidly  becoming  a  thing  of 
the  past  for  American  carriers.  This  is  a  result  of  a 
decrease  in  the  number  of  surplus  pilots  available,  rapid 
expansion  throughout  the  airline  industry,  and  forced 
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retirement  of  a  large  number  of  Vietnam  era  pilots.  This 
imbalance  in  the  supply  and  demand  of  surplus  military 
pilots  is  forcing  the  American  carriers  to  hire  increasing 
numbers  of  civilian  trained  pilots  and  may  eventually  force 
them  into  hiring  pilots  for  ab-inito  training. 
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III.  HISTORY  OF  PILOT  SELECTION  RESEARCH 


Orville  and  Wilbur  Wright  employed  the  initial  pilot 
selection  measure  to  determine  who  would  be  the  first  to 
test  their  new  invention,  the  airplane.  The  selection 
measure  they  chose  was  a  simple  flip  of  a  coin.  Wilbur  won 
the  toss,  but  due  to  mechanical  problems  his  flight  lasted 
less  than  four  seconds  and  covered  less  than  100  feet.  The 
Flyer  was  repaired,  and  five  days  later  it  was  Orville' s 
turn.  His  flight  off  that  small  hill  in  Kitty  Hawk  on  17 
December,  1903  is  generally  recognized  as  the  watershed 
event  that  ushered  in  the  age  of  flight (Cope,  1996). 

Aviation  technology  has  evolved  greatly  since  this  historic 
day  nearly  a  century  ago,  and  so  have  the  needs  and  methods 
of  pilot  selection. 

The  literature  on  pilot  selection  is  largely  focused  on 
military  pilots  due  to  the  military  requirement  to  train 
large  numbers  of  pilots,  especially  during  times  of  war,  and 
the  considerable  resources  available  to  conduct  research. 
Burke  &  Hunter (1990)  noted  during  a  recent  study  on  pilot 
selection  that  of  254  research  studies  found,  only  seven 
dealt  with  the  selection  of  pilots  for  non-military 
settings.  The  vast  literature  on  pilot  selection  can  be 
generalized  into  four  main  categories:  paper-and-pencil 
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general  cognitive  measures,  psychomotor  measures, 
personality  measures,  and  job  sample  measures.  These 
categories  are  explored  in  turn  below. 

A.  GENERAL  COGNITIVE  MEASURES 

The  field  of  aviation  psychology  was  born  during  World 
War  I  as  an  offshoot  of  the  general  military  recruit  testing 
program.  When  the  United  States  entered  the  war,  the  armed 
forces  had  fewer  than  500  pilots  within  their  ranks,  but  by 
the  wars  end  the  number  exceeded  19, 000 (Griff in  &  North, 

1977) .  During  this  tremendous  buildup  there  was  a 
requirement  to  provide  a  nearly  continuous  stream  of  new 
pilots  to  meet  the  demands  of  the  war  effort.  Staggering 
attrition  rates  during  training,  often  as  high  as  50  to  90 
percent  in  the  early  years  of  the  war,  drew  the  attention  of 
military  leaders  and  scientists (Hunter  &  Burke,  1995). 

In  an  effort  to  reduce  the  high  levels  of  attrition, 
scientifically  designed  tests  were  constructed  to  select 
pilot  candidates.  These  earliest  tests,  designed  by 
Bachman ( 1918)  and  Stratton,  McComas,  Coover,  and 
Bagby(1920),  were  very  rudimentary  by  today's  standards. 
Without  previous  knowledge  to  draw  upon,  these  scientists 
designed  tests  for  abilities,  such  as  judgment  of  distance, 
speed,  and  time,  they  thought  would  be  important  to  flying. 

Bachman  designed  tests  for  naval  aviators  which 
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involved  the  estimation  of  the  length  of  sticks,  the 
relative  speed  of  revolving  disks,  and  the  time  for  sand  to 
flow  from  one  container  to  another.  Stratton,  et  al., 
refined  these  tests  by  including  tasks  such  as  judgment  of 
curves  and  route  memory  though  the  use  of  a  cardboard  maze 
in  the  selection  of  Army  Air  Service  pilots.  It  was 
determined  that  these  tests  were  correlated  with  flight 
training  success. 

The  United  States  was  the  only  country  to  use  general 
intelligence  testing  as  a  means  of  pilot  selection  during 
World  War  I.  Hermon(1919)  conducted  a  study  using  the 
Thorndike  Intelligence  Test  in  the  selection  of  Army  Air 
Service  Pilots.  This  test  was  given  to  150  pilots  divided 
into  three  groups  of  50  with  the  following  classifications: 
flying  ability  rated  as  very  good,  flying  ability  rated  as 
very  bad,  and  unknown  flying  ability.  A  correlation  of  .35 
was  found  between  the  Thorndike  Intelligence  Test  and  these 
rough  measures  of  flying  ability. 

In  a  later  review  of  this  study.  Hunter  and  Burke (1990) 
note  that  the  use  of  extreme  groups  inflated  the  observed 
correlations  and  the  statistical  significance  could  not  be 
calculated.  The  study  is  however  an  early  indication  of  a 
relationship  between  general  intelligence  and  flying 
ability.  the  military's  interest  in  pilot  selection 
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research  waned  in  the  post-war  period  as  many  of  the 
psychologists  who  had  been  involved  in  this  research 
returned  to  private  practice. 

As  World  War  II  loomed,  U.S.  interest  in  pilot 
selection  research  was  renewed.  Building  on  the  theme  of 
general  intelligence  testing,  and  expanding  on  it  to  include 
mechanical  comprehension,  tests  were  designed  under  the 
Civilian  Pilot  Training  Program,  sponsored  by  the  Civil 
Aeronautics  Administration.  These  tests  along  with  an 
aviation  classification  test  were  later  adopted  by  both  the 
Army  and  the  Navy  in  the  selection  of  pilot  candidates.  The 
aviation  selection  test  adopted  by  the  Navy  eventually 
became  the  first  version  of  the  Aviation  Selection  Test 
Battery (ASTB) . 

Fiske(1947)  evaluated  these  tests  on  three  separate 
samples  of  Navy  pilot  candidates  in  order  to  validate  their 
effectiveness.  He  reported  that  mechanical  comprehension 
tests  had  consistently  higher  correlation  with  flight 
performance  than  did  general  intelligence  as  measured  by 
Wonderlic  s  Personnel  Test.  Melton (1947)  found  similar 
correlation  for  mechanical  comprehension  in  a  study  of  U.S. 
Army  Air  Corps  pilots  candidates. 

Some  20  years  later,  Berkshire (1967)  tried  to  identify 
critical  attributes  that  the  selection  battery  failed  to 
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assess  in  a  study  of  naval  aviation  candidates  who  had 
scored  high  on  the  Navy  selection  battery,  but  failed  to 
complete  training.  His  study  produced  three  new  tests: 
altitude  judgment,  maneuver  tests,  and  instriucient 
comprehension.  Subseguent  validation  tests  were  conducted 
for  altitude  judgment  and  instrument  comprehension. 

Berkshire  failed  to  find  correlation  for  flight  training 
success  and  the  altitude  judgment  test  but  revealed  a 
correlation  for  the  instrument  comprehension  test. 

Berkshire  found,  as  Fiske(1947)  and  Melton(1947)  had 
earlier,  that  the  best  single  predictor  of  success  in  his 
test  battery  was  the  mechanical  comprehension  test. 

In  1971  the  Navy  revamped  the  selection  test  battery. 
The  individual  components  with  weighted  scores  were  combined 
to  form  a  set  of  composite  scores  known  as  the  Academic 
Qualifying  Test  (AQT)  and  the  Flight  Aptitude  Rating (FAR) . 
These  composite  scores,  with  minor  modifications  under  the 
current  ASTB,  remain  the  foundation  upon  which  student  naval 
aviators  are  selected.  North  and  Griffin (1977)  conducted  an 
analysis  of  the  1973  cohort  of  Navy  pilot  trainees  which 
found  significant  correlation  for  individual  components  of 
the  FAR  in  the  areas  of  mechanical  comprehension,  spatial 
apperception,  and  biographical  inventory. 

Like  the  Navy,  both  the  Army  and  Air  Force  have  an 
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extensive  battery  of  written  tests  used  for  pilot  selection. 
The  Army  has  the  Flight  Aptitude  Selection  Test (FAST),  a 
compilation  of  eight  subtests  on  biographical  information, 
instrument  comprehension,  mechanical  principles,  complex 
movements,  stick  and  rudder  orientation,  helicopter 
information,  and  flight  planning.  And  finally,  the  Air 
Force  has  the  Air  Force  Officer  Qualifying  Test(AFOQT). 

Olea  and  Rea (1993)  and  Carretta  and  Rea (1994)  have  closely 
examined  the  predictive  validity  of  the  most  recent  version 
of  the  AFOQT.  These  studies  have  explored  the  composition 
of  the  various  subtests  in  terms  of  general  cognitive 
ability,  (g) ,  and  specific  ability, (s) .  As  was  previously 
found,  g-loaded  sections  of  the  AFOQT  were  the  best 
predictors  of  success  in  flight  training. 

The  use  of  pencil-and-paper  tests  of  cognitive  ability 
in  pilot  selection  is  not  confined  to  the  United  States. 
Burke (1993)  reports  that  tests  of  spatial  orientation, 
mathematical  and  verbal  reasoning,  instrument  comprehension, 
mechanical  principles  and  general  intelligence  are  widely 
used.  NATO  countries  as  well  use  some  or  all  of  these  tests 
in  their  pilot  selection  process.  While  these  test  have 
shown  little  improvement  in  prediction  validity  since  they 
were  first  introduced  during  WWII,  they  remain  the  backbone 
of  U.S.  pilot  selection  measures  because  of  their  relatively 


30 


low  cost  per  individual  and  the  ease  of  administration  at 
decentralized  testing  centers. 

B .  PSYCHOMOTOR  MEASURES 

The  use  of  apparatus-based  testing  measures  as  a  means 
of  pilot  selection  has  existed  since  World  War  I.  They  have 
however  experienced  periods  of  disfavor,  especially  in  the 
United  States,  due  to  problems  associated  with  calibrating 
the  machinery  and  decentralized  testing  sites.  Early  tests 
conducted  during  World  War  I  were  rather  rudimentary  in 
nature  and  lacked  a  significant  scientific  foundation  as  a 
basis  for  inclusion  in  the  selection  process.  Henmon (1919) , 
Stratton,  et  al.(1920),  and  Dockery  and  Isaacs (1921) 
performed  tests  which  measured  simple  reaction  times,  but 
these  tests  indicated  this  was  not  a  valid  predictor  of 
pilot  ability (Burke,  1990) . 

Studies  conducted  during  and  after  World  War  II  to 
evaluate  pilot  performance  saw  great  improvement  in  the  area 
of  scientific  rigor.  Melton(1947)  conducted  a  series  of 
tests  for  the  U.S.  Air  Force  which  evaluated  the  validity  of 
the  Complex  Coordination  Test  in  predicting  pilot  success. 
This  test,  built  upon  earlier  research  by  Reid (1924)  and 
Mashburn  (1934) ,  used  an  apparatus  which  was  configured  to 
resemble  an  aircraft  and  consisted  of  four  groups  of 
electric  lamps  surrounding  a  speed  indicator.  These  lamps 
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would  illuminate  to  indicate  the  simulated  orientation  of 
the  aircraft  when  it  departed  from  straight  and  level 
flight.  The  goal  was  to  manipulate  the  stick  and  rudder  to 
return  to  straight  and  level  flight  thereby  extinguishing 
the  lamps.  Using  this  test,  correlation  with  a  dichotomous 
pass/fail  criteria  on  completion  of  training,  ranged  from 
0.19  to  0.45. 

Fleishman (1956)  conducted  similar  tests  to  which  he 
added  the  Rudder  Control  Test,  Two-handed  Coordination  Test, 
Directional  Control  Test,  and  Rotary  Pursuit  Test  for  which 
he  found  significant  correlation  with  completion  of  flight 
training.  His  conclusion  was  that  the  Complex  Coordination 
Test  was  the  best  of  these  psychomotor  tests  as  a  predictor 
of  pilot  training  success.  Despite  the  early  promise  of 
psychomotor  tests,  the  Navy  never  utilized  these  tests  in 
the  selection  of  aviators.  It  was  Navy  policy  not  to  use 
test  devices  unless  they  could  be  administered  easily  and 
inexpensively  at  decentralized  test  stations (North  & 

Griffin,  1977) .  The  Air  Force  abandoned  its  apparatus 
testing  program  in  the  early  1950 's  because  of  problems  with 
maintaining  and  calibrating  test  equipment. 

The  dawn  of  the  computer  age  has  given  new  life  to  the 
use  of  psychomotor  testing  in  pilot  selection.  The  most 
recent  computer-based  test  developed  by  the  Air  Force  is  the 
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Basic  Attributes  Test (BAT).  This  new  device  was  designed  to 
use  microprocessor  technology  in  the  administration  of  the 
Two-hand  Coordination  Test  and  the  Complex  Coordination 
Test.  The  BAT  was  successfully  validated  by  tests  at  the 
Air  Force  Human  Resources  Laboratory,  and  has  been  expanded 
to  12  subtests.  Kantor  and  Carretta  (1988)  conducted  a 
cross-validation  of  the  BAT,  that  produced  a  correlation  of 
0.18  using  a  dichotomous  pass/fail  variable.  The  practical 
utility  of  the  test  was  demonstrated  by  showing  that  using 
the  10'"*’  percentile  as  a  cut-off  score  would  effectively 
reduce  attrition  by  20%.  It  would  also  however  reject  8%  of 
graduates . 

Blower  and  Dolgin(1991)  continued  the  Navy's  effort  to 
identify  valid  psychomotor  tests.  An  evaluation  of  a 
recently  developed  test  battery  designed  by  the  Naval 
Aerospace  Medical  Research  Lab,  reports  significant 
correlation  results  for  three  of  the  tests.  Absolute 
Difference-Horizontal  Tracking,  Complex  Visual  Information, 
and  Risk  Taking.  These  test  were  found  to  be  generally 
equivalent  in  the  prediction  of  pilot  training  success  and 
they  recommend  that  any  one  of  the  three  be  used  in 
conjunction  with  the  ASTB  to  increase  the  validity  of  the 
selection  model. 

Hunter  and  Burke (1994)  found  that  psychomotor  and  other 
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apparatus-based  tests  are  among  the  best  predictors  of  pilot 
training  success.  As  computer  technology  advances  it  is 
likely  that  more  realistic,  portable,  and  cost-effective 
computer-based  psychomotor  tests  will  evolve. 

C .  PERSONALITY  MEASURES 

The  use  of  personality  measures  as  a  means  of  pilot 
selection  has  met  with  only  limited  success.  Early  attempts 
suffered  from  the  same  conditions  that  afflicted  early 
psychomotor  studies,  namely  lack  of  scientific  rigor. 

Rippon  and  Manual (1918)  described  the  successful  pilot  as  a 
"sportsman  who  seldom  takes  his  work  seriously  but  looks 
upon  Hun-strafing  as  a  great  game."  Dockery  and 
Isaacs (1921)  thought  "quiet,  methodical  men  are  among  the 
best  fliers."  These  assessments  reflected  the  general 
belief  that  underlying  personality  characteristics  were  in 
some  way  associated  with  becoming  a  successful  pilot (Burke, 
1990)  . 

Some  success  with  the  use  of  personality  measures  came 
during  World  War  II  with  the  development  of  biographical 
inventories,  the  use  of  which  continues  today  as  part  of  the 
AFOQT  and  the  Navy  ASTB.  In  a  review  of  Navy  selection 
research,  Griffin  and  Mosko(1977)  found  almost  40  different 
personality  measures  that  had  been  evaluated  and  failed  to 
make  significant  contributions  to  the  selection  process. 
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They  attributed  much  of  the  failure  to  test  bias.  All  the 
studies  reviewed  involved  the  selection  of  student  naval 
aviators,  a  group  they  contend  is  highly  intelligent  and 
susceptible  to  response  faking.  Dolgin  and  Gibb (1989)  also 
comment  on  the  failure  of  personality  measures,  attributing 
the  failure  to  methodological  problems  in  the  test.  They 
contend  that  these  tests  were  designed  to  test  heterogeneous 
groups,  while  military  aviation  candidates  tend  to  be  a 
homogeneous  group.  Also,  given  that  the  training  is 
voluntary,  it  may  only  attract  certain  personality  types. 

Despite  a  lack  of  success  in  this  area,  the  Navy  and 
Air  Force  continue  to  investigate  the  use  of  computer-based 
personality  measurement  tests  in  the  pilot  selection 
process.  Helton (1993)  evaluated  a  pilot  personality 
questionnaire  for  the  Navy  and  found  that  one  of  12  scales, 
assertiveness,  correlated  significantly  with  pass/fail  in 
pilot  training. 

D.  JOB  SAMPLE  MEASURES 

A  job  sample  test  is  an  artificially  created  situation 
where  the  individual  being  tested  is  required  to  perform  a 
range  of  the  functions  that  would  actually  be  required  on 
the  job (Hunter  and  Burke,  1995).  Until  the  early  1970 's, 
job  sample  tests  for  pilots  were  usually  conducted  in  light, 
single  engine  aircraft.  Boyle  and  Hagin(1953), 
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Bigbee (1954) ,  and  Ambler  and  Wallace  (1967)  conducted  studies 
of  pilot  candidates  who  received  light  plane  instruction 
prior  to  entering  the  training  pipeline.  Each  of  these 
studies  found  a  correlation  between  the  light  plane 
instruction  and  later  performance  in  flight  training.  The 
Air  Force  continues  to  use  light  plane  evaluation  measures 
in  its  pilot  selection  while  the  Navy  has  phased  them  out 
due  to  budgetary  constraints. 

As  computers  become  more  powerful  and  accessible,  less 
expensive  simulators  are  being  used  in  job  sample  measures. 
Hill  and  Goebel (1971)  developed  a  simulator-based  job  sample 
measure  using  a  Link  General  Aviation  Trainer (GAT-1) .  This 
device  underwent  a  series  of  modifications  and  eventually 
became  the  Automated  Pilot  Aptitude  Measurement 
System  (APAMS )  .  Hunter  and  Thompson  (1978)  used  the  APAMS  to 
conduct  two  studies  of  pilot  candidates  for  which  they 
reported  a  correlation  between  APAMS  and  later  flight 
performance.  In  the  civilian  sector.  Stead (1991)  reported 
on  the  validity  of  a  simulator  check-ride  conducted  by 
Qantas  in  the  B747.  He  found  that  performance  on  the 
simulator  check-ride  correlated  significantly  with 
performance  on  all  training  criteria. 
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E.  INTEGRATING  THE  RESEARCH  FINDINGS 

A  technique  that  allows  the  integration  of  findings 
from  multiple  research  studies  and  produces  a  single 
correlation  that  efficiently  describes  the  validity  results 
from  a  number  of  studies  is  called  meta-analyis .  Hunter  and 
Burke (1994)  have  applied  this  technique  to  studies  of  pilot 
selection.  Their  results  are  presented  in  Table  3-1.  As 
can  be  seen  from  the  table,  job  sample  measures  were  found 
to  be  among  the  best  predictors,  followed  closely  by  gross 
dexterity.  In  interpreting  their  results.  Hunter  and  Burke 
make  the  following  conclusions.  First,  if  the  lower  95% 
confidence  limit  is  positive,  this  leads  to  the  conclusion 
that  the  mean  sample  weighted  correlation  across  studies  is 
assumed  to  be  non-zero  but  may  be  influenced  by  moderator 
variables.  Second,  if  the  lower  95%  confidence  limit  is 
negative  while  the  upper  95%  confidence  limit  is  positive, 
this  leads  to  uncertainty  about  the  whether  the  true  value 
of  the  mean  sample  weighted  correlation  is  other  than  zero. 
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Table  3-1 

Meta-analysis  Results 


PREDICTOR 

r; 

mean 

Nx 

^95 

U95 

General  ability 

0.13 

14 

-0.05 

0.30 

Verbal  Ability 

0.12 

17 

-0.09 

0.33 

Quant.  Ability 

0.11 

34 

0.01 

0.21  ! 

Spatial  Ability 

0.19 

37 

0.05 

0.32  ' 

Mechanical 

0.29 

36 

0.11 

0.48  ; 

General  Info. 

0.25 

13 

0.06 

0.44  ^ 

Aviation  Info. 

0.22 

23 

0.06 

0.38 

Gross  Dexterity 

0.32 

60 

0.15 

0.49 

Fine  Dexterity 

0.10 

12 

-0.09 

0.29 

Perceptual  Speed 

0.20 

41 

0.05 

0.35 

Reaction  Time 

0.28 

7 

0.16 

0.39 

Bioinventory 

0.27 

21 

0.07 

0.47 

Age 

-0.10 

9 

-0.25 

0.05  i 

Education 

0.06 

9 

-0.16 

0.27  ' 

Job  Sample 

0.34 

16 

0.19 

0.55  i 

Personality 

0.10 

46 

-0.16 

0.37 

r^ean  Mean  sample  weighted  correlation 

N35  Number  of  studies 

L95  Lower  95%  confidence  interval 

U95  Upper  95%  confidence  interval 
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Pilot  selection  has  been  and  continues  to  be  of  great 
interest  to  both  military  and  civilian  researchers.  Despite 
over  70  years  of  research,  an  optimal  selection  method  has 
yet  to  evolve.  While  there  have  been  significant  advances  in 
the  selection  measures  used  since  their  inception  during 
World  War  I,  only  a  little  over  half  of  the  variance  in 
pilot  performance  can  be  explained  by  the  current  measures. 

General  cognitive  tests  continue  to  be  the  foundation 
of  both  the  Navy  and  Air  Force  selection  processes  because 
they  provide  the  best  "bang  for  the  buck."  It  has  been 
suggested  however,  that  because  there  has  been  very  little 
improvement  in  the  validity  of  these  measures  since  World 
War  II,  they  have  come  about  as  far  as  they  can  as  a 
prediction  measure.  Ease  of  administration  and  the 
relatively  low  cost  per  individual  are  the  primary  reasons 
for  the  continued  dominance  of  this  type  of  assessment. 

The  answer  to  improved  validity  in  the  pilot  selection 
process  most  likely  lies  in  the  areas  of  psychomotor  and  job 
sample  measures.  The  rapid  increase  in  technology  that  we 
have  experienced  in  the  last  few  years  has  led  to  the 
development  of  incredibly  realistic  computer-based 
simulators.  These  simulators,  only  dreamed  of  a  decade  ago, 
should  provide  military  researchers  an  ideal  platform  to 
design  valid  psychomotor  and  job  sample  measures.  These 
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simulators  are  not  inexpensive  to  build  and  operate,  but 
given  the  escalating  costs  of  attrition  from  military  flight 
training,  even  a  marginal  improvement  in  the  attrition  rate 
could  lead  to  substantial  savings.  These  savings  combined 
with  the  falling  cost  of  computer  technology  may  soon  make 
these  measures  cost  effective. 
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IV.  CONSTRUCTION  OF  PROBABILITY  MODEL 


A.  DATA  SOURCES 

The  data  set  for  this  analysis  was  obtained  from  three 
separate  sources.  Data  on  flight  school  performance  was 
obtained  from  the  Naval  Operational  Medical  Institute (NOMI) 
located  in  Pensacola,  Florida.  These  data  included  flight 
performance  scores.  Aviation  Selection  Test  Battery  scores, 
attrition  data,  academic  performance,  and  dates  of  entry  and 
completion  of  the  flight  program. 

Demographic  and  Naval  Academy  admissions  data  was 
obtained  from  records  provided  by  the  Naval  Personal 
Research  and  Development  Center (NPRDC)  in  San  Diego, 
California.  And  finally,  data  on  Academy  performance 
including  academic  and  military  quality  point  ratings.  Order 
of  Merit,  and  academic  major  was  obtained  from  the  Naval 
Academy  Office  of  Institutional  Research  in  Annapolis, 
Maryland.  These  data  sources  were  matched  by  social 
security  number  and  verified  by  midshipman  identification 
number . 

The  database  that  resulted  from  this  merge  contained 
observations  on  2,340  Naval  Academy  midshipmen  from  the 
classes  of  1995  and  1996.  Several  constraints  were  placed 
on  the  original  data  set  in  order  to  isolate  those 
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observations  to  be  included  in  the  study.  First,  those 
individuals  who  did  not  graduate  were  removed,  leaving  1765 
cases.  Second,  those  individuals  who  were  not  selected  for 
aviation  training  were  excluded.  This  reduced  the  data  set 
to  711  observations.  Third,  due  to  the  inherent  differences 
in  the  training  curriculum.  Naval  Flight  Officers  were 
eliminated.  This  reduced  the  data  set  to  495  observations. 
Fourth,  student  pilots  who  were  still  in  the  training 
pipeline,  which  resulted  in  incomplete  grades,  were 
eliminated.  And  finally,  student  pilots  who  were 
immediately  disqualified  from  training  during  their  initial 
flight  physical  were  removed.  This  left  a  total  of  272 
observations  in  the  data  set  for  this  analysis. 

B.  ANALYSIS  OF  VARIABLES 

For  this  analysis  a  two-stage  Heckman  procedure  is 
utilized  requiring  both  a  probit  and  ordinary  least-squares 
model.  The  variables  for  these  models  were  analyzed  as 
follows : 

1 .  Dependant  Variables 

The  dependent  variable  used  in  the  first-stage  probit 
model  was  ADVANCE.  This  variable  takes  on  a  value  of  zero 
for  those  individuals  who  failed  to  complete  either  the 
aviation  indoctrination  or  primary  flight  training  phases 
and  a  value  of  1  for  those  individuals  who  completed  through 
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the  primary  flight  training  stage  and  moved  on  to  advanced 
training.  The  data  indicate  that  87.1%  of  Naval  Academy 
graduates  completed  the  primary  stage  of  flight  training 
while  12.9%  attrited  from  the  program. 

The  dependent  variable  for  the  second-stage  ordinary 
least-squares  model  was  FLIGHTGRD.  This  is  the  cumulative 
average  score  for  the  54  sorties  that  comprise  the  primary 
flight  training  syllabus.  This  score  is  derived  from  the 
flight  instructor  rating  based  on  a  4.0  scale.  Each 
maneuver  performed  during  a  given  sortie  is  graded  as  one  of 
the  following:  above  average (4 . 0) ,  average (3 . 0) ,  below 
average (2 . 0) ,  or  unsatisfactory (0 . 0) .  The  distribution  of 
these  scores  is  displayed  below  in  Figure  4-1. 
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2 .  Independent  Variables 

The  selection  of  independent  variables  was  determined 
from  previous  research.  Descriptive  statistics  for  these 
variables  are  displayed  in  Table  4-1.  Each  of  these 
variables  and  their  expected  impact  on  the  models  is 
discussed  below.  Dummy  variables  were  created  to  represent 
minority  status,  marital  status,  gender,  previous  flight 
experience,  academic  major,  change  in  major,  and  service. 

MINORITY-  This  variable  was  created  to  indicate 
minority  status.  Based  on  demographic  data  from  NPRDC,  the 
majority  of  individuals  in  the  study,  91.2%,  are  Caucasian 
and  are  coded  as  zero.  Due  to  the  relatively  small 
representation  of  individual  minority  groups,  all 
individuals  that  did  not  fall  into  the  majority  group  were 
combined  into  a  single  minority  group.  This  group 
represented  8.8%  of  the  data  and  are  coded  as  1.  Previous 
research  has  shown  that  a  smaller  percentage  of  minority 
candidates  meet  the  minimum  standards  of  the  selection 
tests.  For  this  reason  the  expected  sign  of  this 
coefficient  is  negative. 

MARRIED-  This  variable  indicates  marital  status  and 
takes  on  a  value  of  1  if  the  individual  is  married  and  zero 
otherwise.  Because  being  married  places  extra  demands  upon 
the  student  pilot,  the  expected  sign  is  negative. 
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Table  4-1  Descriptive  Statistics 


N 

Minimum 

Maximum 

Mean 

Std. 

Deviation 

ADVANCE 

272 

.00 

1.00 

.8713 

.3355 

FLTGRADE 

237 

2.991 

3.124 

3.06584 

2.58E-02 

AGE 

272 

21.50 

27.50 

23.3971 

1.1062 

AQPR 

272 

208.00 

396.00 

292.4963 

42.2504 

MQPR 

272 

268.00 

387.00 

332.6471 

25.5421 

SATM 

272 

471.00 

790.00 

631.5000 

63.2247 

TIS 

272 

311.00 

714.00 

511.5588 

92.7015 

AQR 

272 

3 

9 

6.46 

1.46 

BIPILOT 

272 

4 

9 

7.61 

1.32 

PFAR 

272 

4 

9 

6.16 

1.43 

DELTAMAJ 

272 

.00 

1.00 

5.5E-02 

.2287 

GENDER 

272 

.00 

1.00 

7.4E-d2 

.2615 

MAJOR 

272 

.00 

1.00 

.6838 

.4658 

MARINE 

272 

.00 

1.00 

.1397 

.3473 

MARRIED 

272 

.00 

1.00 

2.2E-02 

.1471 

MINORITY 

272 

.00 

1.00 

8.8E-02 

.2842 

PREFLY 

237 

.00 

1.00 

5.5E-02 

.2282 

Valid  N 

(listwise) 

237 

Note:  The  notation  E  represents  the  operation  of  taking 
a  numerical  value  to  a  particular  power.  For  example 
3.14E-02  equals  3.14x10-2  or  0.0314. 
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GENDER-  This  variable  was  created  to  separate  males 
from  females.  Males,  who  represent  92.6%  of  the  sample,  are 
coded  zero.  Females,  representing  7.4%,  are  coded  1. 
Selection  tests  and  performance  evaluations  are  designed  to 
be  gender  neutral  so  it  is  unclear  what  the  sign  of  this 
coefficient  will  be. 

PREFLY-  This  variable  is  created  to  separate  those 
individuals  who  had  previous  flight  experience  prior  to 
entry  into  flight  training.  Information  for  this  variable 
is  self  reported  by  the  student  aviators.  Based  on  personal 
experience,  it  is  suggested  that  a  minimum  threshold  of 
flight  experience  must  be  obtained  before  this  experience 
becomes  meaningful.  An  arbitrary  value  of  20  hours  was 
selected  as  the  threshold  for  this  variable.  Individuals 
with  less  than  20  hours  of  experience  are  coded  as  zero. 
Those  with  more  than  20  hours  are  coded  as  one.  The 
expected  sign  for  this  coefficient  is  positive. 

MAJOR-  This  variable  separates  those  individuals  who 
graduated  with  a  technical  degree,  a  so-called  group  1  or 
group  2  major  at  the  Academy,  from  those  who  graduated  with 
a  non-technical  degree.  Technical  majors  are  coded  as  one 
and  non-technical  majors  as  zero.  Technical  degrees  involve 
a  higher  level  of  math  and  engineering  comprehension  than 
non-technical  majors.  For  this  reason  the  expected  sign  for 


47 


this  coefficient  is  positive. 

DELTAMAJOR-  This  variable  identifies  those  individuals 
who  changed  from  a  technical  major  to  a  non-technical  major. 
This  change  would  indicate  that  these  individuals  were 
having  difficulty  with  the  more  demanding  levels  of  math  and 
engineering  associated  with  a  technical  degree.  The 
expected  sign  on  this  coefficient  is  negative. 

MARINE-  This  variable  is  created  to  distinguish  Marine 
aviators  from  naval  aviators.  Marine  aviators  must  attend 
The  Basic  School  prior  to  reporting  to  Pensacola.  There 
they  learn  tactics  and  basic  Marine  doctrine.  It  is 
suggested  that  the  additional  military  experience  that  these 
individuals  receive  will  make  them  less  likely  to  attrite 
from  training  than  their  Navy  counterparts.  For  this  reason 
the  expected  sign  for  this  coefficient  is  positive. 

AGE-  This  variable  is  the  student  naval  aviator's  age 
at  the  beginning  of  flight  training.  This  variable  was 
derived  by  subtracting  as  student's  date  of  birth  from  their 
Aviation  Indoctrination  class  convening  date.  Previous 
research  indicates  that  age  has  a  negative  influence  during 
flight  training.  The  expected  sign  on  this  coefficient  is 
negative . 

BIPILOT-  This  score  is  derived  from  the  biographical 
information  section  of  Aviation  Selection  Test  Battery 
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(ASTB) .  The  raw  score  is  converted  into  a  standard  nine- 
point  scale.  This  portion  of  the  test  is  specifically 
designed  to  predict  attrition  from  flight  training.  The 
higher  the  score,  the  less  likely  an  individual  is  to 
attrite.  For  this  reason  the  expected  sign  on  this 
coefficient  is  positive. 

PFAR-  This  score  is  the  pilot  flight  aptitude  from  the 
ASTB  and  is  designed  to  predict  basic  flight  performance. 
Based  on  a  standard  nine-point  scale,  a  higher  score  should 
reflect  better  flight  performance.  The  expected  sign  on 
this  coefficient  is  positive. 

AQPR-  This  score  is  the  individual's  academic  quality 
point  rating  or  grade  point  average  for  academic  performance 
while  at  the  Naval  Academy.  The  expected  sign  on  this 
coefficient  is  positive. 

MQRP-  This  variable  represents  the  individual's 
military  quality  point  rating,  a  measure  of  military 
performance  while  at  the  Naval  Academy.  The  expected  sign 
this  coefficient  is  positive. 

TIS-  This  variable  is  the  score  from  the  Strong- 
Campbell  Technical  Interest  Survey  taken  as  part  of  the 
Naval  Academy  admissions  process.  The  expected  sign  on  this 
coefficient  is  positive. 

SATM-  This  variable  is  the  average  score  achieved  on 
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the  math  section  of  the  Scholastic  Aptitude  Test  reported 
during  the  Naval  Academy  admissions  process.  A  higher  score 
indicates  higher  mathematical  ability.  The  expected  sign  on 
this  coefficient  is  positive. 

AQR-  This  variable  is  the  individual's  academic 
quality  rating  from  the  ASTB.  Based  on  a  nine  point 
standard  scale,  this  score  is  designed  to  predict  academic 
performance  during  flight  training.  The  expected  sign  on 
this  coefficient  is  positive. 

C .  MODEL  DESCRIPTION 

The  model  specification  used  to  describe  the 
determinants  of  a  Naval  Academy  graduate's  flight  grades 
takes  the  following  form: 

Fi  =  f(gi,Pi,ti,Si,Zi)+  Ui.  4-1 

Fi  is  the  flight  grade  of  the  ith  individual,  gi  is  a 
measure  of  cognitive  ability,  p^  a  measure  of  psychomotor 
ability,  ti  personality  measures,  Si  job  sample  measures, 
represents  other  factors  affecting  performance  such  as  race 
an  gender,  and  Ui  is  a  random  disturbance  term  reflecting 
unobserved  ability  characteristics  and  the  measurement  error 
of  performance  statistics. 

It  is  assumed  that  this  function  will  take  the  form  of 
the  linear  equation  Yi=a+bXi+ei,  where  Y^,  the  dependent 
variable,  represents  flight  grades  and  the  right  hand 
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term  represents  a  vector  of  independent  variables.  Usually# 
when  trying  to  explain  or  predict  changes  in  due  to  X^, 
ordinary  least-sguares,  which  minimizes  the  vertical  sum  of 
the  squared  deviation  from  the  fitted  line#  is  used.  The 
ordinary  least-squares  procedure  assumes  the  following: 

•  The  relationship  between  Y  and  X  is  linear. 

•  The  Xi' s  are  nonstochastic  variables  whose  values 
are  fixed. 

•  The  error  term  has  zero  expected  value  and 
constant  variance  for  all  observations. 

•  The  random  variables  Si  are  statistically 
independent . 

•  The  error  term  is  normally  distributed. 

When  these  assumptions  are  satisfied#  estimators  are 
unbiased  and  consistent.  The  normality  assumption  permits 
hypothesis  testing  to  be  conducted  when  the  sample  size  is 
relatively  small. 

1 .  Selectivity  Bias 

The  term  bias  refers  to  the  potential  misestimate  of 
the  effect  of  a  treatment  on  the  outcome.  Suppose  for 
example  that  one  is  studying  the  wages  of  women  or  the 
automobile  purchasing  behavior  of  a  random  sample  of  the 
population.  It  is  possible  to  know  the  actual  wages  of 
those  women  who  are  working#  but  it  is  not  possible  to  know 
the  reservation  wage  for  those  who  are  not.  In  the 
automobile  example#  for  those  who  happened  to  purchase  a 
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car,  the  expenditure  can  be  recorded.  But  for  both  those 
who  did  and  did  not  purchase  a  car,  one  cannot  measure  the 
maximum  amount  they  would  have  been  willing  to  pay  for  the 
automobile. 

In  both  of  the  preceding  examples,  the  dependent 
variable  is  censored,  that  is,  information  is  missing  for 
the  dependent  variable,  but  the  corresponding  information 
for  the  independent  variables  is  present.  This  situation  is 
referred  to  as  selectivity  bias.  A  similar  situation  exists 
in  our  model  of  flight  grades  in 'that  flight  grades  exist 
only  for  those  who  were  able  to  complete  the  program.  It  can 
be  shown  that  ordinary  least-squares  estimation  of  the 
censored  regression  model  may  fail  to  satisfy  the  assumption 
that  the  error  term  of  the  model  has  zero  expected  value 
and,  if  this  is  so,  would  generate  biased  and  inconsistent 
parameter  estimates.  In  order  to  correct  for  the 
selectivity  bias,  however,  a  relatively  simple  two-stage 
estimation  procedure  that  yields  consistent  parameter 
estimates  can  be  employed. 

2.  Censored  Regression  Model 

This  section  will  show  how  the  ordinary  least-squares 
estimation  of  a  censored  regression  model  can  generate 
biased  and  inconsistent  parameter  estimates.  We  will  then 
describe  how  to  correct  for  this  situation.  As  mentioned 
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before,  in  the  censored  regression  model  information  for 
some  observations  of  the  dependent  variable  are  missing 
while  corresponding  information  for  the  independent 
variables  is  present. 

Suppose  that  the  underlying  continuous  version  of  the 
model  is  given  by: 

Fi  =  |3o+3iXii+32Zi+u^i,  4-2 

where  represents  the  final  flight  grade  for  those 
individuals  who  completed  the  primary  stage  of  flight 
training.  For  those  individuals  who  did  not  complete,  this 
value  equals  zero.  Assume  that  there  is  a  survival  equation 
that  must  be  equal  to  or  greater  than  some  minimiim  score  (C) 
for  Fi  to  be  observed.  The  survival  equation  is  a  function 
of  flight  performance  (Fi) ,  academic  performance  (Ai) ,  and 
military  performance  (Mi)  and  takes  the  form  of: 

Ysi=aFi+bAi+cMi+Ui ,  4-3 

where^ 

Fi=3o+PlXli+32Zi+UFi, 

Ai=ao+aiX2i+a2Zi+UAi,  and 

Mi=Yo+YiX3i+Y2Zi+UMi . 

In  these  equations  the  Xji(j=l,2,3)  variables  represent  a 
vector  of  variables  that  are  unique  to  the  individual 


^In  reality  each  of  these  equations  may  have  its  own  minimum  score. 
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equation.  The  variables  represent  a  vector  of  variables 
common  to  the  three  equations. 

Let 

Oo  +  tto  +  Yo  -C)=  00, 

(^2  +  «2  +  Yz)=  Qir  and 

+  'J’Ai  +  +  Ui)  =  e^. 

Equation  4-3  can  now  be  expressed  as  follows: 
if  Ysi=  6+3iXii+aiX2+YiX3i+9iZi+ei^0  then  di=l 

4-4 

if  Y3i=  e+3iXii+aiX2+YiX3i+0iZi+ei<O  then  di=0 . 

This  implies  that 

P(di=l)=P (advance)  =F(0+3iXii+aiX2i+YiX3i+eiZi)  ,  4-5 

where  F  is  a  cumulative  normal  probability  function. 

Returning  to  equation  4-2  and  evaluating  given  the 
sample  selection  rule  in  equation  4-4, 

E  (FjXii,  Zi,  Ysi-CsO)=(3o+PaXii+p2Zi+E  (uFi|Xii,  Zi,  Ygi-C^O)  . 
James  Heckman  shows  that^ 

E  (UpilXii ,  Zi ,  Ygi-C^O)  =  (ajYs/ 4-6 

where  is  the  covariance  (UpUyg)  and  is  the  standard 
deviation  (Yg)  . 


^Heckman,  James  J.,  “The  Common  Structure  of  Statistical  Models  of  Truncation,  Sample  Selection,  and 
Limited  Variables  and  a  Simple  Estimator  for  Such  Models”,  Annals  of  Economic  and  Social 
Measurement,  1976. 

^Each  of  the  individual  equations  used  to  create  equation  4-3  would  require  a  probit  equation  to  calculate 
an  individual  X  term.  For  modeling  purposes  these  terms  are  summed  into  an  aggregate  X. 
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Equation  4-2  can  now  be  written  as 

(“^FYs/ *^Ys)  •  4-7 

Heckman  has  shown  that 

Xi  =  f(e+piXii+aiX2+YiX3i+eiZi)/  4-8 

F(e+piXii+aiX2+YiX3i+eiZi) , 

where  f  is  the  probability  density  function  of  a 
standardized  normal  variable,  and  F  is  the  corresponding 
cumulative  distribution  function.  If  we  have  estimates  of 
Ai,  we  can  use  them  to  adjust  for  the  fact  that  the  expected 
value  of  the  censored  error  term  shown  in  equation  4-6  may 
not  equal  to  zero.  The  Heckman  two-stage  process  is  used  to 
yield  consistent  estimates  of  Po/  r  Pa* 

D.  TWO- STAGE  HECKMAN  PROCEDURE 

In  the  first-stage,  A^  is  estimated  utilizing  the 
probit  model; 

Pi=F(0+(3iXii+aiX2+YiX3i+eiZi)  ,  4-9 

where  Pi  equals  the  probability  of  completing  primary  flight 
training.  This  probit  model  is  estimated  with  a  maximum- 
likelihood  technique  by  distinguishing  those  observations 
for  which  Fi>0  from  those  for  which  Fi=0.  From  the 
estimated  parameters  of  the  probit  model,  Ai  may  be 
calculated  using  equation  4-8. 

The  second  stage  of  the  two-stage  estimator  uses  the 
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following  ordinary  least-squares  model: 

Fi=|3o+3iXi+32Zi+aJ^i+Vi,  4-10 

where  has  been  added  as  an  additional  explanatory 
variable  and  is  an  error  term  with  expected  value  equal 
to  zero  in  large  samples.  Because  Jii  approaches  as  the 
sample  size  gets  large,  and  X^  normalizes  the  mean  to 
zero,  ordinary  least-squares  estimation  of  equation  4-10 
yields  consistent  estimates  of  Po/  Pi/  arid  Pj- 
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V.  ANALYSIS  OF  EMPIRICAL  MODELS 


This  section  contains  the  analysis  of  the  multivariate 
models  discussed  in  the  previous  chapter.  Using  the  Heckman 
two-stage  regression  technique  which  combines  the 
multivariate  probit  model  with  the  (adjusted)  ordinary 
least-squares (OLS)  regression  model  we  examine  the 
individual  effects  of  the  independent  variables  on  flight 
performance  holding  several  other  variables  constant.  The 
first-stage  probit  model  is  designed  to  create  a  new 
independent  variable,  K,  which  is  included  in  the  second- 
stage  OLS  model  as  an  explanatory  variable  to  correct  for 
potential  selectivity  bias. 

During  the  analysis,  two  specifications,  an  initial  and 
final  specification,  were  created  for  both  parts  of  the  two- 
stage  procedure.  In  the  initial  specification  all  relevant 
independent  variables  are  included.  In  the  alternate 
specification,  the  first-stage  probit  model  is  altered  to 
include  only  those  variables  with  a  "t'^'-statistic^  greater 
than  one.  Also  retained  are  the  gender  and  minority 
variables  which  have  t-statistics  less  than  one  but  are  of 
interest  to  the  study.  The  second-stage  alternate 

^The  calculated  “t”  is  equal  to  the  estimated  coeflScient  divided  by  the  standard  error.  This  value 
approaches  the  /-statistic  in  large  samples. 
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specification  also  retained  only  those  variables  with  a 
t-statistic  greater  than  one  and  a  few  select  variables  of 
interest  to  the  study.  Additionally,  the  OLS  model  was  run 
twice  for  the  final  specification.  In  the  first  estimation, 
the  model  included  the  independent  variable.  In  the 
second  estimation,  this  variable  is  eliminated  to  display 
the  results  without  an  adjustment  for  selectivity  bias. 

A.  FIRST-STAGE  PROBIT  MODEL 

The  initial  specification  for  the  first-stage  model  is 
as  follows: 

Prob(  advance  )=  F(3o+  PiBIPILOT  +  P2PFAR  +  33AQPR  + 
P^DELTAMAJOR  +  P5MAJOR  +  pgMARINE  +  P7AGE  + 

PgGENDER  +  pgMARRIED  +  PiqMINORITY  +  P^MPOR  + 

P12AQR  +  P13TIS  +  P14SATM) 

where  F  is  the  cumulative  normal  probability  function  of  the 
underlying  random  process. 

The  results  for  the  initial  probit  specification  are 
displayed  in  Table  5-1. 
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Table  5-1 

Probit  Analysis  of  Likelihood  of  Naval  Academy  Graduates 
Completing  Primary  Stage  of  Flight  Training:  Initial 

Specification 


INDEPENDENT  VARIABLE 

BIPILOT . 

PFAR . 

AQPR . 

DELTAMAJOR . 

MAJOR . 

MARINE . 

AGE . 

GENDER . 

MARRIED . 

MINORITY . 

MPQR . 

AQR . 

TIS . 

SATM . 

INTERCEPT . 

SAMPLE  SIZE . 


COEFFICIENT 
ESTIMATE 
..  0.21577 
..  0.13094 
..  0.01064 
. .-0.53254 
..  0.57910 
..  0.80868 
..  0.7483 
..  0.2715 
. .-0.74831 
. .-0.26249 
. .-0.00451 
. .-0.09065 
. .-0.00155 
. .-0.00273 
. .-3.78041 
.272 


T-STATISTIC 

2.91967*** 

0.90455 

2.54256** 

-2.42007** 

2.30031** 

1.85285* 

0.65925 

0.61111 

-1.13623 

-0.75288 

-0.73788 

-0.62157 

-1.15827 

-1.15185 

-1.00691 


CHI-SQUARE  (df=258)=  289.090,  significance  =.089 
***  SIGNIFICANT  AT  0.01  LEVEL 

**  SIGNIFICANT  AT  0.05  LEVEL 

*  SIGNIFICANT  AT  0.1  LEVEL 


Many  of  the  variables  in  Table  5-1  are  not 
statistically  significant.  However,  the  chi-square 
statistic  is  significant  which  indicates  that  the  model  is  a 
fairly  good  fit.  The  variables  BIPILOT,  AQPR,  DELTAMAJOR, 
MAJOR,  and  MARINE  are  all  significant  with  coefficients  that 
match  their  predicted  sign.  The  AGE  variable,  which  was 
expected  to  have  a  negative  coefficient,  has  the  opposite 
sign  in  this  model  but  is  not  statistically  significant. 
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The  alternate  specification  was  developed  by  taking  the 
significant  variables  in  the  initial  specification  as  a  base 
model.  Then  to  allow  for  the  possibility  of 
multicolinearity  in  the  original  specification,  the  other 
variables  were  individually  added  back  into  the  model  and 
the  change  in  the  chi-square  statistic  caused  by  the 
addition  of  that  variable  was  analyzed.  Variables  that 
raised  the  value  of  the  chi-square  statistic  were  retained 
in  the  model,  while  those  that  reduced  the  value  were 
omitted.  The  goal  was  to  build  the  best  possible  first- 
stage  model  giving  proper  account  to  theory, 
multicolinearity,  and  tests  of  significance.  The  regression 
results  for  the  alternate  specification  are  shown  in  Table 
5-2. 
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Table  5-2 

Probit  Analysis  of  Likelihood  of  Naval  Academy  Graduates 
Completing  Primary  Stage  of  Flight  Training:  Alternate 

Specification 


INDEPENDENT  VARIABLE 

COEFFICIENT 

T-STATISTIC 

ESTIMATE 

BIPILOT . 

.  0.20461 

2.91743*** 

AQPR . 

DELTAMAJOR . 

major . 

.  0.00839 

. -0.51660 

.  0.43444 

2.72541*** 

-2.42801** 

1.96295** 

MARINE . 

.  0.72423 

1.82212* 

age . 

.  0.08938 

0.84737 

GENDER . 

.  0.24898 

0.61072 

MINORITY . 

. -0.23999 

-0.70480 

INTERCEPT . 

. -5.30902 

-1.72949 

SAMPLE  SIZE . 

.  . . .272 

CHI-SQUARE  (df=263)=  393.595,  significance  =  .001 
***  SIGNIFICANT  AT  0.01  LEVEL 

**  SIGNIFICANT  AT  0.05  LEVEL 

*  SIGNIFICANT  AT  0.1  LEVEL 

All  variables  in  the  alternate  specification  retain  the 
same  sign  as  in  the  initial  specification.  Additionally, 
the  significance  level  of  all  variables,  with  the  exception 
of  AQPR,  did  not  change.  The  significance  level  of  AQPR 
increased  to  the  0.01  level  in  the  alternate  specification. 
The  goodness  of  fit  chi-square  statistic  shows  an  increase 
in  the  alternate  specification,  rising  from  289.090  (df=258) 
in  the  initial  specification  to  393.595  (df=263)  in  the 
alternate  specification.  The  decrease  in  p-value  associated 

with  this  change  in  chi-square  is  from  0.089  to  0.001^ 


^The  p-value  is  the  probability  of  obtaining  a  chi-square  value  at  least  as  large  as  the  calculated  value, 
when  the  null  hypothesis  is  true. 
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1.  Goodness  of  Fit 


An  important  measure  of  quality  of  the  individual 
probit  equations  is  the  Pearson  goodness  of  fit  chi-square 
statistic.  The  chi-square  statistic  from  each  probit 
equation  was  compared  against  the  critical  value.  This 
statistic  accepts  the  null  hypothesis  that  the  explanatory 
variables  in  the  model  do  not  contribute  to  the  overall 
equation  if  the  computed  significance  level  does  not  exceed 
the  critical  value.  In  the  initial  specification  the  chi- 
square  value  was  large  enough  to  reject  the  null  hypothesis 
at  the  0.089  level  and  in  the  alternate  specification  the 
chi-square  value  was  large  enough  to  reject  the  null 
hypothesis  at  the  0.001  level.  The  chi-square  value  in  the 
alternate  specification  indicates  that  the  variables 
retained  in  the  alternate  specification  contribute  more  to 
the  quality  of  the  equation  than  those  in  the  initial 
specification. 

2 .  Notional  Pilot 

Although  the  primary  purpose  of  the  first-stage  probit 
model  was  to  generate  the  variable  in  order  to  estimate 
consistent  and  unbiased  results  in  the  second-stage  OLS 
model,  it  is  interesting  to  interpret  the  results  of  the 
probit  regressions.  The  coefficients  of  the  variables  are 
converted  to  an  estimate  of  the  change  in  probability  of 
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completing  primary  flight  training  for  a  notional  pilot  who 
possesses  a  given  set  of  characteristics.  The  notional 
pilot  was  defined  by  setting  all  the  dummy  variables  to  zero 
and  by  using  the  average  value  of  the  continuous  variables. 
Once  the  "base-case"  pilot  was  defined,  each  variable  was 
manipulated  individually  to  generate  the  change  in 
probability  of  completing  primary  flight  training  associated 
with  a  change  in  that  independent  variable®.  The  notional 
pilot  has  the  following  characteristics: 

•  Navy 

•  Caucasian 

•  Male 

•  Non-technical  degree 

•  Did  not  change  out  of  a  technical  major 

•  AQPR  of  2.92 

The  change  in  probability  associated  with  changes  in  these 
characteristics  are  displayed  in  Table  5-3.  The  change  in 
AQPR  from  2.92  to  3.02  was  chosen  to  illustrate  the  change 
associated  with  a  0.1  change  in  AQPR. 


®This  is  an  approximation  based  on  the  method  found  in  Pindyck  and  Rubinfeld. 
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Table  5-3  Change  in  Probability 


Variable 

Initial  Value 

New  Value 

Delta  Prob. 

AQPR 

2.92 

3.02 

Increase  3.4% 

MARINE 

No 

Yes 

Increase  15% 

MAJOR 

Non- technical 

Technical 

Increase  11% 

DELTAMAJOR 

No 

Yes 

Decrease  18% 

BIPILOT 

5 

6 

Increase  16% 

B.  SECOND-STAGE  OLS  MODEL 

The  second-stage  of  the  Hecknam  regression  technique  is 

the  construction  of  an  ordinary  least-squares  model  that 

includes  the  as  an  independent  explanatory  variable  to 

correct  for  potential  selectivity  bias.  The  initial 

specification  of  this  model  takes  the  following  form: 

Fi  =  3o  +  PiBIPILOT  +  (32PFAR  +  P3AQPR  +  P4DELTAMAJOR  + 
PsMAJOR  +  pgMARINE  +  3,AGE  +  pgGENDER  +  pgMARRIED  + 
PioMINORITY  +  PiiMPQR  +  P12AQR  +  PijTIS  +  Pi^SATM  + 

PisPREFLY  +  +  Vi. 

The  regression  results  for  this  model  are  displayed  in 
Table  5-4. 
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Table  5-4 

Regression  Results  for  Naval  Academy  Graduate's  Flight 
Grades:  Initial  Specification 


DEPENDENT  VARIABLE : FLIGHT  GRADES 

INDEPENDENT  VARIABLE  COEFFICIENT  T-STATISTIC 


BIPILOT . 

PFAR . 

AQPR . 

DELTAMAJOR.  . 

MAJOR . 

MARINE . 

AGE . 

GENDER . 

MARRIED . 

MINORITY. . . . 

MQPR . 

AQR . 

TIS . 

SATM . 

PREFLY . 

LAMDA . 

INTERCEPT . . . 
SAMPLE  SIZE. 
R-SQUARE .... 
ADJ  R-SQUARE 
F-STATISTIC. 


ESTIMATE 
.-0.097 
.  0.570 
.  0.012 
.  0.296 
.  0.17 
.-0.154 
.-0.136 
.-0.614 
.  1.10 
.-0.241 
.  0.012 
.-0.0016 
.  0.0024 
.  0.0032 
.  1.790 
.-1.182 
299.34 
237 

. .0.378 
. .0.336 
. .8.962 


-0.0691 

4.9590*** 

2.2957** 

0.7328 

0.4832 

-0.2934 

-0.7603 

-1.0674 

1.0043 

-0.4373 

1.6161* 

-0.7570 

1.4930* 

0.1132 

2.7221*** 

-1.0082 

41.531 


***  SIGNIFICANT  AT  0.01  LEVEL 
**  SIGNIFICANT  AT  0.05  LEVEL 
*  SIGNIFICANT  AT  0 . 1  LEVEL 
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As  can  be  seen  from  the  regression  results,  the 
variables  PFAR,  AQPR,  PREFLY,  TIS,  and  MQPR  are 
statistically  significant  in  this  model.  Several  variables 
that  were  significant  in  the  first-stage  probit  model, 
MARINE,  DELTAMAJOR,  and  MAJOR  are  not  statitically 
significant  in  the  second-stage  OLS  model.  This  is 
consistent  with  the  theory  used  in  formulating  equation  4-3, 
that  the  overall  survival  equation  is  the  sum  of  multiple 
equations  acting  simultaneously  and  that  some  of  the 
variables  are  only  germaine  to  individual  parts  of  the 
overall  equation. 

In  the  interest  of  parsimony,  only  those  variables 
significant  at  the  0.10  level,  along  with  the  MINORITY, 
GENDER,  and  variables  are  retained  for  the  alternate 
specification.  The  alternate  specification  is  as  follows: 

Po  +  PiPFAR  +  32APQR  +  PsGENDER  +  (34MINORITY  + 

PsMARRIED  +  PgMQPR  +  P^TIS  +p8PREFLY  +  +  Vi. 

The  regression  results  for  this  model  are  displayed  in 
Table  5-5.  The  variable  is  not  significant  in  the 
alternate  specification  and  there  is  actually  a  decrease  in 
the  t-statistic  in  the  alternate  model.  The  non¬ 
significance  of  this  variable  leads  to  the  conclusion  that 
selectivity  bias  is  not  pervasive  for  this  sample  of  flight 
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students.  A  possible  explanation  is  that,  while  the  data  is 
censored,  the  covariance  between  the  survival  equation  and 
the  flight  grade  equation  is  fairly  small.  If  this  is  the 
case,  the  OLS  model  without  the  variable  is  a  more 
appropriate  model  for  this  sample.  A  comparison  of  the 
alternate  specification  with  and  without  the  variable 
shows  that  while  there  are  minor  changes  in  the  t- 
statistics,  none  of  the  variables  change  significance  levels 
and  the  signs  on  the  coefficients  remain  the  same. 

As  a  result  of  the  non-significant  the  alternate 
specification  is  run  without  the  correction  for  selectivity 
bias.  The  results  for  this  regression  are  displayed  in 
Table  5-6.  For  ease  of  data  interpretation,  the  Beta 
coefficient  is  included  in  Tables  5-5  and  5-6.  The  Beta 
coefficient  is  a  normalized  coefficient  that  can  be  used  to 
assess  the  relative  importance  of  the  independent  variables 
in  the  model . 
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Table  5-5 

Regression  Results  for  Naval  Academy  Graduate's  Flight 
Grades:  Alternate  Specification 


*** 

** 

* 


DEPENDENT  VARIABLE : FLIGHT  GRADES 

INDEPENDENT  VARIABLE  COEFFICIENT  T-STATISTIC  BETA 

ESTIMATE 

PFAR .  0.580  5.529***  0.331 


AQPR  .  0 

GENDER . -0 

MARRIED .  0 

MINORITY . -0 

MQPR .  0 

TIS .  0 

PREFLY .  1 

lAMDA . -0 

INTERCEPT . 294 

SAMPLE  SIZE . 237 

R- SQUARE . 0 

ADJ  R- SQUARE . 0 

F-STATISTIC . 15 


SIGNIFICANT 

AT 

O.OI 

LEVEL 

SIGNIFICANT 

AT 

0.05 

LEVEL 

SIGNIFICANT 

AT 

0.1 

LEVEL 

014 

3.139*** 

0.219 

461 

-0.872 

-0.047 

914 

0.930 

0.051 

357 

-  0.683 

-0.038 

012 

1.696* 

0.117 

0028 

1.880* 

0.101 

683 

2.706** 

0.149 

977 

-0.966 

-0.055 

535 

123.349 

375 

350 

129 
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Table  5-6 

Ordinary  Least-squares  Regression  Results  for  Naval  Academy 
Graduate's  Flight  Grades:  Alternate  Specification  Without 
Sample  Selection  Correction 

DEPENDENT  VARIABLE : FLIGHT  GRADES 

INDEPENDENT  VARIABLE  COEFFICIENT  T-STATISTIC  BETA 


ESTIMATE 

PFAR .  0.603  5.851***  0.342 

AQPR  .  0.013  3.092***  0.215 

GENDER . -0.425  -0.805  -0.044 

married .  0.748  0.773  0.042 

MINORITY . -0.468  -0.916  -0.049 

MQPR .  0.013  1.817*  0.125 

TIS .  0.0028  1.843*  0.099 

PREFLY .  1.657  2.666**  0.146 

INTERCEPT . 284.535  125.759 

SAMPLE  SIZE . 237 

R-SQUARE . 0.372 

ADJ  R-SQUARE . 0.350 

F-STATISTIC . 16.908 


SIGNIFICANT 

AT 

0.01 

LEVEL 

SIGNIFICANT 

AT 

0.05 

LEVEL 

SIGNIFICANT 

AT 

0.1 

LEVEL 

The  regression  results  of  the  alternate  specification 
in  Table  5-5  show  that  the  coefficient  sign  for  all 
variables  remains  the  same  as  in  the  initial  specification. 
The  PFAR  variable  is  significant  at  the  0.01  level  and  shows 
that  a  higher  score  on  the  pilot  flight  aptitude  rating  of 
the  Aviation  Selection  Test  Battery  predicts  higher  flight 
grades.  The  Beta  coefficient  indicates  that  an  increase  of 
one  standard  deviation  in  the  PFAR  score  would  result  in  a 
0.342  standard  deviation  increase  in  the  final  flight  grade. 
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Another  variable  significant  at  the  0.01  level  is  AQPR. 
Table  5-5  shows  that  a  higher  academic  quality  point  rating 
while  at  the  Academy  predicts  higher  flight  grades.  The 
Beta  coefficient  indicates  that  a  one  standard  deviation 
increase  in  AQPR  leads  to  a  0.215  standard  deviation 
increase  in  the  final  flight  grade.  PREFLY  is  significant 
at  the  0.05  level  showing  that  previous  flight  experience  is 
a  significant  predictor  of  higher  final  flight  grades.  MQPR 
is  marginally  significant  at  the  0.1  level  showing  that  a 
higher  military  quality  point  rating  predicts  higher  flight 
grades.  TIS  is  also  marginally  significant  at  the  0.1  level 
indicating  that  a  higher  score  on  the  Strong-Campbell 
Technical  Interest  Survey  predicts  higher  flight  grades. 

The  remaining  variables,  including  GENDER  and  MINORITY, 
contained  in  the  alternate  specification  were  not 
statistically  significant. 

In  the  alternate  specification  the  variables  PFAR, 

AQPR,  MPQR,  and  TIS  each  show  incremental  improvement  in 
their  t-statistic,  while  PREFLY  shows  a  modest  decrease  but 
remains  significant  at  the  0.05  level.  The  adjusted  R- 
squared  for  the  model,  which  explains  the  amount  of  variance 
accounted  for  by  the  model,  increased  from  0.336  in  the 
initial  specification  to  0.35  in  the  alternate 
specification.  This  leads  to  the  conclusion  that  the 
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alternate  specification  does  a  somewhat  better  job  of 
explaining  the  variance  in  the  data  when  the  degrees  of 
freedom  of  the  model  are  taken  into  account. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  thesis  investigates  the  relationship  between 
observable  characteristics  and  performance  in  primary  flight 
training  using  a  two-stage  Heckman  regression  procedure. 

The  hypotheses  identified  in  the  introduction  will  now  be 
restated  and  compared  with  the  results  of  this  procedure. 

Hypothesis  1. 

•  Naval  Academy  graduates  who  score  higher  on  the 
biographical  inventory  are  more  likely  to  complete 
primary  flight  training  than  those  with  lower 
scores . 

Based  on  the  significance  level  of  the  BIPILOT  variable 
in  the  probit  model,  the  null  hypothesis  that  the 
coefficient  on  this  variable  is  zero  is  rejected.  Rejection 
of  the  null  hypothesis  supports  the  alternate  hypothesis 
that  the  true  value  of  this  coefficient  is  not  zero.  This 
result  indicates  that  there  is  a  relationship  between  higher 
scores  on  the  biographical  inventory  and  successful 
completion  of  primary  flight  training. 

Hypothesis  2. 

•  Naval  Academy  graduates  who  score  higher  on  the 
pilot  flight  aptitude  rating  achieve  higher  flight 
grades  that  those  with  lower  scores. 
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The  significance  of  the  PFAR  variable  in  the  second- 
stage  OLS  equation  leads  us  to  reject  the  null  hypothesis 
that  the  value  of  this  coefficient  is  zero  in  favor  of  the 
alternate  hypothesis  that  the  true  value  of  this  coefficient 
is  not  zero.  This  indicates  that  there  is  a  relationship 
between  higher  scores  on  the  flight  aptitude  rating  and 
flight  grades. 

Hypothesis  3. 

•  Naval  Academy  graduates  with  higher  academic 
quality  point  ratings  achieve  higher  flight  grades 
than  those  with  lower  ratings . 

The  significance  of  the  AQPR  variable  in  the  second- 
stage  OLS  model  allows  the  null  hypothesis  that  the  value  of 
this  coefficient  is  zero  to  be  rejected  in  favor  of  the 
alternate  hypothesis  that  the  value  is  not  zero.  This 
indicates  that  there  is  a  direct  relationship  between  the 
level  of  academic  achievement  at  the  academy  and  primary 
flight  grades. 

Hypothesis  4. 

•  Naval  Academy  graduates  with  higher  military 
quality  point  ratings  achieve  higher  flight  grades 
than  those  with  lower  ratings . 

The  significance  of  the  MPQR  variable  is  inconclusive. 
The  null  hypothesis  that  there  is  no  relationship  is 
rejected  at  the  0.1  level  but  not  the  0.05  level. 


Hypothesis  5. 

•  Naval  Academy  graduates  with  technical  degrees 
achieve  higher  scores  that  graduates  with  non¬ 
technical  degrees. 

The  results  of  the  probit  model  indicate  that  graduates 
with  technical  degrees  are  more  likely  to  finish  the  primary 
stage  of  training.  The  null  hypothesis  that  the  coefficient 
of  this  variable  in  the  second  stage  OLS  is  equal  to  zero, 
however,  cannot  be  rejected.  This  provides  evidence  that 
the  type  of  degree  obtained  from  the  academy  is  not  related 
to  primary  flight  grades. 

Hypothesis  6. 

•  Male  graduates  achieve  higher  flight  grades  than 
female  graduates. 

Based  on  the  significance  level  of  the  GENDER  variable, 
the  null  hypothesis  that  the  value  of  the  coefficient  is 
different  than  zero  cannot  be  rejected.  This  indicates  that 
there  is  no  difference  associated  with  gender  in  primary 
flight  grades. 

Hypothesis  7. 

•  Ethnic  majority  graduates  achieve  higher  flight 
grades  than  minority  graduates. 

The  null  hypothesis  that  the  value  of  the  MINORITY 
coefficient  is  different  from  zero  cannot  be  rejected.  This 
indicates  that  there  is  no  difference  in  primary  flight 
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grades  associated  with  the  race  of  the  individual. 

Hypothesis  8. 

•  Naval  Academy  graduates  with  previous  flight 
experience  achieve  higher  flight  grades  than 
graduates  with  no  previous  flight  experience. 

The  value  of  the  PREFLY  variable  is  significant  at  the 
0.05  level  allowing  us  to  reject  the  null  hypothesis  that 
the  coefficient  is  zero.  Rejection  of  the  null  hypothesis 
supports  the  alternate  hypothesis  that  the  value  of  this 
coefficient  is  not  zero,  indicating  that  primary  flight 
grades  are  positively  influenced  by  previous  flight 
experience. 

Based  on  the  results  of  hypothesis  testing,  the 
conclusions  of  this  study  are  summarized  as  follows: 

•  The  biographical  inventory  section  of  the  ASTB  is 
a  valid  predictor  of  primary  flight  training 
completion. 

•  The  pilot  flight  aptitude  rating  of  the  ASTB  is  a 
valid  predictor  of  primary  flight  performance. 

•  Academic  performance  at  the  Academy  is  a  valid 
predictor  of  primary  flight  performance. 

•  Previous  flight  experience  is  a  valid  predictor  of 
primary  flight  performance. 

These  findings  are  consistent  with  the  findings  of  the 
body  of  research  contained  in  the  literature  review.  An 
additional  conclusion  of  this  study  is  that  selectivity  bias 
is  not  a  problem  with  this  sample  of  flight  students.  There 
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are  however  several  other  potential  sources  of  bias  in  the 
statistical  estimates.  Assignment  to  aviation  duty  is  a 
done  on  a  volunteer  basis.  It  is  therefore  possible  that 
there  is  problem  with  self-selection  bias  in  the  first-stage 
probit  analysis.  Individuals  who  have  little  or  no  interest 
in  aviation  careers  or  who  are  deterred  by  the  significant 
commitment  incurred  by  those  who  attend  flight  training  can 
choose  not  to  participate  in  the  selection  process.  There 
is  also  a  possible  problem  associated  with  the  range 
restriction  of  the  variables  used  in  the  models.  Because 
some  of  these  variables  are  used  as  selection  measures,  the 
range  that  the  variable  can  assume  is  artificially  limited. 
Omitted  variables  may  also  cause  a  problem.  In  addition  to 
the  variables  included  in  the  model,  there  may  exist  other 
relevant  variables  that  have  not  been  measured  or  for  which 
there  is  no  reliable  proxy.  Omission  of  these  variables 
might  lead  to  bias  through  model  specification.  It  is  hoped 
that  through  continued  research  in  the  area  of  pilot 
selection,  these  and  other  problems  can  eventually  be 
addressed. 

B.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

As  a  result  of  this  research  several  interesting 
avenues  for  future  research  have  been  identified.  For 
example,  it  might  be  interesting  to  study  factors  that 
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predict  success  for  Naval  Flight  Officers.  Do  factors  that 
predict  success  of  pilots  also  predict  success  for  Naval 
Flight  Officers? 

The  literature  review  revealed  that  there  has  been 
limited  success  in  the  use  of  personality  measures  in  the 
selection  of  aviators.  This  has  been  attributed  to  the  fact 
that  many  of  the  student  aviators  are  able  to  "fake"  the 
test  and  provide  the  most  desirable  answers.  A  solution  to 
this  problem  would  be  to  have  a  test  that  was  not  associated 
with  selection  into  an  aviation  program.  The  Academy  has 
access  to  data  of  this  type  in  the  Myers-Briggs  type 
indicator  test  and  related  variants.  These  tests  are 
conducted  early  during  the  midshipmen' s  time  at  the  academy 
and  there  may  be  a  smaller  probability  that  the  midshipman 
will  perceive  his/her  answer  as  being  linked  to  the  aviation 
selection  process. 

One  of  the  original  goals  of  this  study  was  to  analyze 
the  results  of  the  interview  data  obtained  during  the 
service  assignment  process.  Unfortunately  this  information 
no  longer  exists  for  the  graduating  classes  included  in  this 
study.  This  information  does  exist  however  for  subsequent 
classes.  As  these  graduates  complete  flight  training, 
service  assignment  process  information  will  be  available  and 
could  be  used  in  subsequent  analysis. 
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Finally,  as  a  cost  savings  measure,  there  has  been  much 
interest  in  trying  to  identify  and  therefore  eliminate  those 
individuals  who  are  most  likely  to  attrite  from  flight 
training.  But  what  happens  to  those  individuals  who  do 
attrite  from  the  training  program?  Do  these  individuals 
serve  the  minimum  obligation  and  then  leave  the  service?  Do 
they  go  on  to  experience  successful  and  rewarding  careers  in 
some  other  community?  The  human  resource  implications  of 
finding  answers  to  these  questions  could  help  detailers  do  a 
better  job  assigning  individuals  to  future  billets  A  better 
assignment  process  would  help  ensure  that  the  Navy  receives 
the  maximum  return  on  any  further  investment  in  personnel 
capital , 
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